Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 044a8a3122 | |||
| bbaecdb559 | |||
| 35f6584fb2 | |||
| 598c96679a | |||
| 8596877cd9 | |||
| 1d2313b98c |
@@ -1,76 +0,0 @@
|
||||
---
|
||||
name: quality-eval
|
||||
description: Use when the user asks to objectively evaluate, score, rate, audit, or quality-gate code, codebases, files, pull requests, or snippets using a strict 5-dimension engineering rubric with scores and refactoring steps.
|
||||
metadata:
|
||||
short-description: Score code quality with a strict rubric
|
||||
---
|
||||
|
||||
# Quality Eval
|
||||
|
||||
## Role
|
||||
|
||||
Act as a Staff Software Engineer and automated quality gate. Evaluate code objectively against the rubric below, surface hidden anti-patterns, and provide a mathematical grade with atomic refactoring steps.
|
||||
|
||||
## Evaluation Rules
|
||||
|
||||
- Evaluate only against the five rubric dimensions.
|
||||
- Be candid. Do not inflate scores for politeness.
|
||||
- Avoid generic advice. Every recommendation must name a specific code location, behavior, or pattern and include a concrete improvement direction.
|
||||
- Inspect the code before scoring. For codebases, read enough representative files, tests, and architecture boundaries to justify the scope.
|
||||
- When exact line numbers are available, cite them.
|
||||
- Do not reveal private chain-of-thought. In the required `Chain of Thought Analysis` section, provide a concise, step-by-step audit rationale with observable findings and score justifications.
|
||||
|
||||
## Rubric
|
||||
|
||||
Score each dimension from 1 to 5 using these anchors:
|
||||
|
||||
| Dimension | Score 1 (Fail) | Score 3 (Pass) | Score 5 (Exemplary) |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Architecture** | Spaghettified; tight coupling; violated separation of concerns. | Modular but relies on leaky abstractions or mixed domains. | Strict domain isolation; follows SOLID; clear dependency inversion. |
|
||||
| **Readability** | Cryptic naming; deep nesting (>3 levels); widespread DRY violations. | Idiomatic but features over-complex functions or sparse documentation. | Self-documenting; expressive naming; high cohesion; flat structure. |
|
||||
| **Resilience** | Swallows errors blindly; lacks contextual logging; fragile to bad input. | Basic try/catch blocks present but lacks granular, typed error handling. | Explicit error boundaries; contextual logging; structured failure modes. |
|
||||
| **Testability** | Hardcoded dependencies make mocking or isolated testing impossible. | Pure functions are testable, but side-effect heavy logic lacks test hooks. | Decoupled IO; deterministic execution; structured for unit and integration tests. |
|
||||
| **SecOps** | Hardcoded secrets; O(n^2) bottlenecks; zero input sanitization. | Safe from obvious flaws but lacks deep defensive optimization. | Validated inputs; optimized algorithmic complexity; zero security debt. |
|
||||
|
||||
## Scoring Method
|
||||
|
||||
1. Determine the evaluated scope and primary language.
|
||||
2. Identify concrete evidence for each dimension.
|
||||
3. Assign integer dimension scores from 1 to 5.
|
||||
4. Compute `composite_score` as the arithmetic mean of the five dimension scores, rounded to one decimal place.
|
||||
5. Include code snippets only when they make a refactoring step more actionable.
|
||||
|
||||
## Required Output
|
||||
|
||||
Structure every response into exactly these three Markdown sections:
|
||||
|
||||
### 1. Chain of Thought Analysis
|
||||
|
||||
Provide a concise step-by-step audit rationale. Name specific files, functions, patterns, anti-patterns, and rubric anchors. Keep it evidence-based and do not include hidden private reasoning.
|
||||
|
||||
### 2. Normalized Score Report
|
||||
|
||||
```json
|
||||
{
|
||||
"evaluation_metadata": {
|
||||
"target_scope": "string",
|
||||
"primary_language": "string"
|
||||
},
|
||||
"metrics": {
|
||||
"architecture_and_modularity": 0,
|
||||
"readability_and_maintainability": 0,
|
||||
"error_handling_and_resilience": 0,
|
||||
"testability_and_mocking": 0,
|
||||
"security_and_performance": 0
|
||||
},
|
||||
"composite_score": 0.0
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Atomic Refactoring Playbook
|
||||
|
||||
* **High Priority (To lift Score 1/2 to 3):**
|
||||
- [ ] Actionable, specific refactoring step with file/line/context reference.
|
||||
* **Medium Priority (To lift Score 3 to 4/5):**
|
||||
- [ ] Optimization or architectural pattern implementation step.
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
display_name: Quality Eval
|
||||
short_description: Scores code quality with a strict five-dimension rubric and refactoring playbook.
|
||||
default_prompt: Evaluate this code objectively using the quality-eval rubric and return the three-section score report.
|
||||
@@ -30,26 +30,13 @@ the container lifecycle and the copying of skills and env vars into it.
|
||||
"Intended design".
|
||||
- `docs/README.md` — docs overview; when to write which document.
|
||||
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
|
||||
- `docs/research/` — research notes (see `docs/research/README.md`).
|
||||
- `docs/research/` — research notes.
|
||||
- `docs/decisions/` — decision records (ADR-lite).
|
||||
|
||||
## Conventions
|
||||
|
||||
- Three kinds of doc, each with its own conventions in-folder; see
|
||||
`docs/README.md` for when to write which:
|
||||
- **PRDs** (`docs/prds/`) — one feature per file, numbered
|
||||
`NNNN-kebab.md`. A `Status:` line tracks lifecycle: Draft → Active
|
||||
(shipped to `main`) → Superseded/Retargeted. Format in
|
||||
`docs/prds/README.md`.
|
||||
- **Research notes** (`docs/research/`) — opinionated investigations;
|
||||
unnumbered kebab-case, freeform and verdict-first. See
|
||||
`docs/research/README.md`.
|
||||
- **Decision records** (`docs/decisions/`) — ADR-lite, numbered
|
||||
`NNNN-kebab.md`, for policies and non-feature decisions. See
|
||||
`docs/decisions/README.md`.
|
||||
- Keep decision rationale self-contained in the repo, not in Gitea
|
||||
issue threads. Issues are an ephemeral inbox; the durable "why" lives
|
||||
in a PRD, research note, or decision record.
|
||||
- Product requirement docs live in `docs/prds/`.
|
||||
- Research notes live in `docs/research/`.
|
||||
- Low dependencies by default. The project is Python, stdlib-first (no
|
||||
runtime pip dependencies in the package itself; the only language
|
||||
runtime is the Python 3.13 used by the CLI + sidecars). Ask before
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ FROM node:22-slim
|
||||
# tool (curl itself, plus anything that shells out to it) works
|
||||
# against pipelock's bumped TLS without the agent needing local DNS.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install claude-code globally. Pinned to the version verified in the v1
|
||||
|
||||
+2
-2
@@ -6,10 +6,10 @@
|
||||
FROM node:22-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.134.0 \
|
||||
&& npm cache clean --force
|
||||
|
||||
USER node
|
||||
|
||||
+1
-3
@@ -31,7 +31,6 @@
|
||||
# 9099 egress (mitmproxy, pipelock's upstream — not externally
|
||||
# addressed by the agent)
|
||||
# 9418 git-gate (git-daemon)
|
||||
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||
# 9100 supervise (MCP HTTP)
|
||||
|
||||
# Stage 1: pipelock binary. The upstream pipelock image is a
|
||||
@@ -82,7 +81,6 @@ COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
||||
COPY bot_bottle/supervise.py /app/supervise.py
|
||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
||||
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
|
||||
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
|
||||
RUN chmod +x /app/egress-entrypoint.sh
|
||||
|
||||
@@ -99,7 +97,7 @@ RUN mkdir -p \
|
||||
|
||||
# Documentation only — the compose renderer publishes whichever
|
||||
# subset the bottle uses.
|
||||
EXPOSE 8888 9099 9418 9420 9100
|
||||
EXPOSE 8888 9099 9418 9100
|
||||
|
||||
# WORKDIR matches Dockerfile.supervise's prior layout so the
|
||||
# in-app same-dir import in supervise_server.py stays deterministic.
|
||||
|
||||
@@ -6,26 +6,96 @@
|
||||
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||
|
||||
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
||||
Run multiple Claude Code agents on your own machine, each scoped to its own secrets, skills, and egress allowlist.
|
||||
|
||||
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
|
||||

|
||||
|
||||
## Features
|
||||
Four prompts to the agent inside a real bottle:
|
||||
claude replies to `hello there` — proof api.anthropic.com routes
|
||||
through pipelock's bumped TLS end-to-end;
|
||||
asked to GET a non-allowlisted host, the agent's curl gets 403 back
|
||||
from pipelock;
|
||||
asked to POST a credential-shaped body to an allowlisted host, the
|
||||
same 403 — pipelock's DLP body scanner caught it;
|
||||
asked to commit and push an AKIA-shaped key, git-gate's gitleaks
|
||||
pre-receive hook rejects the ref.
|
||||
Run it yourself with `bash scripts/demo.sh`.
|
||||
|
||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
|
||||
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
||||
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
||||
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
|
||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
|
||||
## Why "bot-bottle"?
|
||||
|
||||
Each container is a bottle; Claude is the genie inside. The genie's
|
||||
powers are exactly what the manifest grants it — a specific set of
|
||||
skills, a specific set of secrets, and a specific set of hosts it can
|
||||
reach — nothing more. You uncork one bottle per agent
|
||||
(`./cli.py start <agent>`), many bottles run in parallel, and each is
|
||||
scoped to its task. When the session ends the bottle is destroyed and
|
||||
the genie does not persist.
|
||||
|
||||
## Goals
|
||||
|
||||
- Scope each agent to the minimum credentials and network egress its task actually needs
|
||||
- Run multiple agents in parallel, isolated from each other
|
||||
- Keep code, credentials, and agent activity on infrastructure I control — no third-party agent runtime
|
||||
|
||||
## Project status
|
||||
|
||||
bot-bottle is a self-hosted secure runtime for AI coding agents.
|
||||
Each agent runs in an isolated container or micro-VM-backed bottle with
|
||||
scoped secrets, allowlisted egress, TLS-aware proxying, DLP checks, and
|
||||
a git-gate that withholds upstream credentials and scans pushes before
|
||||
forwarding. The project includes a documented threat model, PRD-driven
|
||||
development history, Docker and smolmachines backends, dashboard and
|
||||
remediation flows, and unit/integration tests covering exfiltration and
|
||||
sandbox escape scenarios.
|
||||
|
||||
## Security model
|
||||
|
||||
Each agent runs in its own bottle: its own container, its own internal
|
||||
Docker network, and its own pipelock sidecar. Bottles don't share
|
||||
state, don't talk to each other, and only get the env vars, skills,
|
||||
SSH identities, and egress hosts the manifest grants them — nothing
|
||||
more. Any one agent only has the access it needs to do its job.
|
||||
|
||||
The bottle limits both what an agent can see and where it can send
|
||||
it. Each bottle gets only the secrets and SSH identities the manifest
|
||||
grants it — a Gitea token but not a GitHub token, a deploy key but
|
||||
not a personal SSH key — so even a compromised or misbehaving agent
|
||||
only handles credentials it was already trusted with for its job.
|
||||
Egress flows through pipelock, which constrains where those
|
||||
credentials can travel: an agent with a Gitea token can reach
|
||||
`gitea.dideric.is`, not arbitrary attacker-controlled hosts. The same
|
||||
constraint blocks DNS-over-HTTPS as an exfil channel — a DoH resolver
|
||||
like `cloudflare-dns.com` would have to be on the allowlist for the
|
||||
agent to reach it at all. The container itself adds a layer between
|
||||
the agent and the host, but the v1 design leans more on secret
|
||||
minimization and egress allowlisting than on the container as a
|
||||
hardened boundary. On Linux hosts where [gVisor](https://gvisor.dev/)
|
||||
is registered with Docker, bot-bottle auto-detects it and launches
|
||||
every bottle under `runsc` for a userspace syscall barrier — no
|
||||
manifest configuration required. The broader v2 discussion lives in
|
||||
`docs/research/stronger-isolation-alternatives.md`.
|
||||
|
||||
The egress proxy and OAuth-token handling below are the load-bearing
|
||||
pieces of v1.
|
||||
|
||||
## Architecture
|
||||
|
||||
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles pipelock + cred-proxy + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||
A bottle is two containers per agent: an `agent` container, and a
|
||||
`sidecars` container that bundles pipelock + egress + git-gate +
|
||||
supervise behind a Python init supervisor (PRD 0024). They share a
|
||||
per-agent Docker `--internal` network; the agent has no default
|
||||
route off-box. All HTTP and HTTPS egress funnels through pipelock,
|
||||
where the egress allowlist, TLS interception, and request-body DLP
|
||||
scanner enforce the manifest before any byte leaves the host. The
|
||||
only egress that doesn't traverse pipelock is git-gate's SSH
|
||||
push/fetch to `bottle.git` upstreams — pipelock can't proxy SSH,
|
||||
so git-gate is its own L4-style egress path with gitleaks doing
|
||||
the pre-receive scan.
|
||||
|
||||
The agent dials the bundle by the legacy short names (`pipelock`,
|
||||
`egress`, `git-gate`, `supervise`); the renderer registers those as
|
||||
docker-network aliases on the bundle so existing HTTPS_PROXY URLs
|
||||
and MCP endpoints resolve without an agent-side change.
|
||||
|
||||
```
|
||||
host ( ./cli.py )
|
||||
@@ -34,21 +104,26 @@ A bottle is two containers per agent: an `agent` container, and a `sidecars` con
|
||||
▼
|
||||
┌─────────────────────────── bottle ──────────────────────────────────┐
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────┐ │
|
||||
│ │ agent image │ HTTP(S) proxy │ cred-proxy │ │
|
||||
│ │ (claude-code, │ ─────────────────►│ (strips/inj │ │
|
||||
│ │ codex, etc) │ │ Authoriz.) │ │
|
||||
│ │ │ └──────┬───────┘ │
|
||||
│ │ environ: URLs │ │ │
|
||||
│ │ only, no real │ ▼ │
|
||||
│ │ tokens │ ┌────────────────┐ │ HTTPS to
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ agent image │ HTTPS_PROXY │
|
||||
│ │ (claude-code, │ ────────────────────────┐ │
|
||||
│ │ built locally) │ │ │
|
||||
│ │ │ plain HTTP │ │
|
||||
│ │ skills, env, │ (token injection) ┌────▼─────────┐ │
|
||||
│ │ ~/.gitconfig, │ ──────────────────►│ cred-proxy │ │
|
||||
│ │ ~/.npmrc, tea │ │ (strips/inj │ │
|
||||
│ │ │ │ Authoriz.) │ │
|
||||
│ │ environ: URLs │ └─────┬────────┘ │
|
||||
│ │ only, no real │ HTTPS_PROXY │ │
|
||||
│ │ tokens │ ▼ │
|
||||
│ │ │ ┌────────────────┐ │ HTTPS to
|
||||
│ │ │ │ pipelock image │──────────┼──► allowlisted
|
||||
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
|
||||
│ │ │ │ body scan, │ │ cred-proxy
|
||||
│ │ │ │ allowlist) │ │ upstreams)
|
||||
│ │ │ └────────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ │ git proxy ┌────────────────┐ │ SSH push/fetch
|
||||
│ │ │ git:// ┌────────────────┐ │ SSH push/fetch
|
||||
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
||||
│ │ │ │ (gitleaks + │ │ upstreams
|
||||
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
||||
@@ -62,25 +137,198 @@ A bottle is two containers per agent: an `agent` container, and a `sidecars` con
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
||||
- **agent image** — built from the provider template Dockerfile
|
||||
(`Dockerfile.claude` for Claude, `Dockerfile.codex` for Codex, or
|
||||
`agent_provider.dockerfile`) on first run; runs the selected agent
|
||||
CLI with the manifest-granted skills, env vars, and `~/.gitconfig`
|
||||
(the latter for the git-gate's `insteadOf` rules when `bottle.git`
|
||||
is set).
|
||||
- **pipelock image** — per-agent sidecar. Terminates the agent's
|
||||
outbound HTTP/HTTPS, enforces the resolved allowlist, runs DLP
|
||||
scanning. Design in `docs/prds/0001-per-agent-egress-proxy-via-pipelock.md`
|
||||
and `docs/prds/0006-pipelock-tls-interception.md`.
|
||||
- **git-gate image** — per-agent sidecar built on `zricethezav/gitleaks`
|
||||
(alpine + gitleaks + git-daemon + openssh-client). Runs
|
||||
`git daemon` over `git://` as a bidirectional mirror of each
|
||||
declared upstream. A pre-receive hook gitleaks-scans incoming
|
||||
refs and forwards clean refs to the real upstream over SSH; an
|
||||
access-hook runs `git fetch origin --prune` against the upstream
|
||||
before every upload-pack so an agent fetch returns whatever the
|
||||
upstream has *now* (fail-closed if unreachable). The agent's
|
||||
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
||||
so push, fetch, clone, and pull all route through. The agent
|
||||
never sees the upstream credential. If the upstream's hostname
|
||||
isn't resolvable from the gate container (e.g. a Tailscale-only
|
||||
host whose public DNS points elsewhere), pin its IP via
|
||||
`ExtraHosts: { "<hostname>": "<ip>" }` on the `bottle.git` entry —
|
||||
the gate's `/etc/hosts` gets the override while the agent's
|
||||
`insteadOf` rewrite still keys off the original hostname. Brought
|
||||
up only when `bottle.git` has entries. Design in
|
||||
`docs/prds/0008-git-gate.md`.
|
||||
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
||||
base, stdlib-only) that holds API tokens declared in
|
||||
`bottle.cred_proxy.routes`. Each route names a `path`,
|
||||
`upstream`, `auth_scheme`, and `token_ref` (host env var); the
|
||||
agent dials `http://cred-proxy:9099<path>...` over plain HTTP
|
||||
and the proxy strips any inbound `Authorization`, injects
|
||||
`<auth_scheme> <token>` using the value held only in its own
|
||||
container's environ, and forwards to the real upstream over
|
||||
HTTPS. SSE responses stream back unbuffered. The cred-proxy's
|
||||
outbound HTTPS routes through pipelock (it trusts pipelock's
|
||||
per-bottle CA), so pipelock's egress allowlist + body scanner
|
||||
apply to cred-proxy traffic the same way they apply to direct
|
||||
agent traffic. Smart-HTTP push paths (`/git-receive-pack`,
|
||||
`/info/refs?service=git-receive-pack`) are refused at the
|
||||
proxy — push must go through `bottle.git` / git-gate where
|
||||
gitleaks runs. Optional per-route `role` tags drive agent-side
|
||||
rewrites: `anthropic-base-url`, `npm-registry`, `git-insteadof`,
|
||||
`tea-login`. The agent's `printenv` shows only proxy URLs —
|
||||
none of the real token values. Design in
|
||||
`docs/prds/0010-cred-proxy.md`.
|
||||
|
||||
When the agent exits, `cli.py` tears down every sidecar that was
|
||||
brought up and the two networks; nothing about a bottle persists
|
||||
between runs.
|
||||
|
||||
## Quickstart
|
||||
|
||||
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
Requires Docker on the host and a long-lived Claude Code OAuth token in
|
||||
your shell env.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
```
|
||||
|
||||
The container is removed automatically when the session ends. If the script
|
||||
is killed with SIGKILL the exit trap won't fire and the container may be
|
||||
left running; remove it with `docker rm -f <container-name>`.
|
||||
|
||||
### Smolmachines backend (experimental, macOS-only)
|
||||
|
||||
A second backend runs the agent in a smolvm micro-VM (libkrun) with the
|
||||
sidecar bundle still in Docker. Selected via
|
||||
`BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>`. Requires
|
||||
`smolvm` on PATH (`curl -sSL https://smolmachines.com/install.sh | sh`).
|
||||
|
||||
The integration tests run against whichever backend the env var
|
||||
selects and skip cleanly when its prerequisites are missing.
|
||||
|
||||
**One-time sudo on first launch (macOS):** smolmachines bottles
|
||||
each reserve a loopback alias from a pool (`127.0.0.16` ..
|
||||
`127.0.0.31`) and bind their bundle's port-forwards to it; the
|
||||
first `./cli.py start` after each reboot prompts for sudo to add
|
||||
missing aliases via `ifconfig lo0 alias`. Aliases persist until
|
||||
reboot; subsequent launches don't prompt. The agent's TSI
|
||||
allowlist is the alias's `/32`, so each bottle can only reach
|
||||
its own bundle's published ports — not other bottles' ports,
|
||||
not other host loopback services (postgres, dev servers, etc.).
|
||||
|
||||
This enforcement requires a workaround for a smolvm 0.8.0 bug:
|
||||
the CLI's `--allow-cidr` flag is silently dropped when combined
|
||||
with `--from <smolmachine>`. The launcher patches smolvm's
|
||||
persistent state DB
|
||||
(`~/Library/Application Support/smolvm/server/smolvm.db`)
|
||||
directly between `machine create` and `machine start` to set
|
||||
the allowlist. The hack falls away automatically when smolvm
|
||||
honors the flag upstream — see the `loopback_alias` module's
|
||||
docstring for the investigation trail.
|
||||
|
||||
## Manifest
|
||||
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
||||
Bottles and agents live as Markdown files with YAML frontmatter under
|
||||
`~/.bot-bottle/`. Each bottle is one file in `bottles/`, each agent
|
||||
is one file in `agents/`:
|
||||
|
||||
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
||||
```
|
||||
~/.bot-bottle/
|
||||
├── bottles/
|
||||
│ ├── dev.md
|
||||
│ └── gitea-dev.md
|
||||
└── agents/
|
||||
├── implementer.md
|
||||
└── researcher.md
|
||||
```
|
||||
|
||||
The filename (without `.md`) is the entity's name. Filenames must
|
||||
match `[a-z][a-z0-9-]*`; files that don't are skipped with a warning.
|
||||
|
||||
A repo can ship its own agent files alongside its code at
|
||||
`<repo>/.bot-bottle/agents/<name>.md`. Those agents reference
|
||||
bottles defined in `~/.bot-bottle/bottles/` (the only place
|
||||
bottles can come from); a `bottles/` subdir in a repo is ignored
|
||||
with a warning. **This is the trust boundary**: bottle infrastructure
|
||||
— credentials, egress allowlists, git remotes — comes from your home
|
||||
directory only. A cloned repo cannot redirect a host env var to an
|
||||
attacker-named upstream because it has no way to declare a bottle.
|
||||
|
||||
### Bottle composition with `extends:`
|
||||
|
||||
A bottle can inherit from another via `extends: <bottle-name>` so
|
||||
operators don't have to duplicate a whole bottle file to vary one
|
||||
field (PRD 0025). The parent's resolved config is the base; the
|
||||
child's declared fields overlay. Merge rules:
|
||||
|
||||
- `env:` — dict merge, child wins on key collision.
|
||||
- `git.user:` — per-field overlay (child's non-empty `name` /
|
||||
`email` wins; empty falls through to parent).
|
||||
- `git.remotes:` — dict merge by host, child wins on host collision.
|
||||
An explicit `git.remotes: {}` clears the parent's remotes; omitting
|
||||
`git.remotes` inherits the parent's remotes.
|
||||
- `agent_provider:`, `egress:`, `supervise:` — full replace when the
|
||||
child declares the field.
|
||||
|
||||
```yaml
|
||||
---
|
||||
extends: dev # inherit everything from bottles/dev.md
|
||||
egress:
|
||||
routes:
|
||||
- host: staging.example.com
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: STAGING_TOKEN
|
||||
---
|
||||
```
|
||||
|
||||
Cycles (`A extends B extends A`), self-references, and missing
|
||||
parents die at parse with a clear pointer. Bottles remain
|
||||
`$HOME`-only — `extends:` preserves the trust boundary above.
|
||||
|
||||
### Provider base bottles
|
||||
|
||||
Keep provider/runtime policy in one home-owned base bottle, then have
|
||||
task bottles extend it. That keeps provider egress/auth in one place
|
||||
without hiding security-relevant routes behind `agent_provider.template`.
|
||||
|
||||
For example, `~/.bot-bottle/bottles/claude.md` can hold the Claude
|
||||
provider selection and Anthropic API egress:
|
||||
|
||||
````markdown
|
||||
---
|
||||
extends: claude # inherit the Claude provider boundary
|
||||
agent_provider:
|
||||
template: claude
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: api.anthropic.com
|
||||
role: claude_code_oauth
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
pipelock:
|
||||
tls_passthrough: true
|
||||
---
|
||||
|
||||
Common Claude provider boundary.
|
||||
````
|
||||
|
||||
Task bottles can then inherit that provider boundary and add their own
|
||||
env/git configuration without repeating the Claude route.
|
||||
|
||||
### Example bottle (`~/.bot-bottle/bottles/gitea-dev.md`)
|
||||
|
||||
````markdown
|
||||
---
|
||||
extends: claude
|
||||
|
||||
env:
|
||||
GIT_AUTHOR_NAME: didericis
|
||||
@@ -95,22 +343,19 @@ git:
|
||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||
KnownHostKey: ssh-ed25519 AAAA...
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
auth:
|
||||
scheme: token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
pipelock:
|
||||
ssrf_ip_allowlist: [100.78.141.42/32]
|
||||
---
|
||||
|
||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||
gitea over SSH for push, token over HTTPS for the API.
|
||||
The `gitea-dev` bottle. Backs my work on personal projects: provider
|
||||
auth through egress and gitea.dideric.is over SSH.
|
||||
````
|
||||
|
||||
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
||||
For a Codex-backed base bottle, set `agent_provider.template: codex`
|
||||
and use the `codex_auth` egress role for the OpenAI API route. The
|
||||
built-in Codex template uses `Dockerfile.codex`; set
|
||||
`agent_provider.dockerfile` to build the agent from a custom
|
||||
Dockerfile while keeping the bot-bottle sidecars in place.
|
||||
|
||||
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
||||
|
||||
````markdown
|
||||
---
|
||||
@@ -122,12 +367,117 @@ skills:
|
||||
You help maintain Gitea-hosted projects.
|
||||
````
|
||||
|
||||
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
The agent's Markdown body is its system prompt (whitespace
|
||||
stripped). The frontmatter declares the bottle to launch in and any
|
||||
skills to mount. You can also include Claude Code subagent fields
|
||||
(`name`, `description`, `model`, `color`, `memory`) in the
|
||||
frontmatter — bot-bottle ignores them at launch but doesn't
|
||||
reject them, so the same file can drop into `~/.claude/agents/` as a
|
||||
Claude Code subagent.
|
||||
|
||||
Unknown top-level frontmatter keys die at load with a "did you mean"
|
||||
pointer; typos don't silently ghost into an empty config.
|
||||
|
||||
The YAML subset the frontmatter accepts is bounded (flat keys,
|
||||
strings / ints / true-or-false bools / null / lists / one-level
|
||||
nested dicts). Anchors, multi-line block scalars, tags, and
|
||||
ambiguous bare strings (`yes` / `NO` / `2026-05-24` /
|
||||
`0x...`) all die with a clear pointer at the spec — quote your
|
||||
strings when in doubt. The full schema lives in
|
||||
`bot_bottle/yaml_subset.py` (~450 lines, stdlib-only, no PyYAML).
|
||||
|
||||
Working examples live under `examples/`. Pipelock's design lives in
|
||||
`docs/prds/0001-per-agent-egress-proxy-via-pipelock.md` and the
|
||||
rationale in `docs/research/pipelock-assessment.md`. The trust
|
||||
boundary rationale lives in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
|
||||
## Auth: Claude OAuth token, not API key
|
||||
|
||||
Bottles that use `agent_provider.template: claude` authenticate
|
||||
`claude` inside the container with the same Pro/Max subscription you
|
||||
already use on the host, via a long-lived OAuth token. No
|
||||
`ANTHROPIC_API_KEY` is needed.
|
||||
|
||||
**Why a token instead of mounting `~/.claude.json`:** on macOS, Claude
|
||||
Code stores OAuth credentials in the encrypted Keychain, not in
|
||||
`~/.claude.json`. Mounting that file into a Linux container does not
|
||||
carry the credentials with it. Linux hosts keep credentials in
|
||||
`~/.claude/.credentials.json`, but to keep the launcher portable
|
||||
bot-bottle uses the env-var path on every host.
|
||||
|
||||
**One-time setup on the host:**
|
||||
|
||||
```sh
|
||||
claude setup-token # browser login, prints a ~1-year OAuth token
|
||||
```
|
||||
|
||||
Stash the token in your shell env (e.g. `~/.zshrc` or a secret manager)
|
||||
as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||
|
||||
```sh
|
||||
export BOT_BOTTLE_CLAUDE_OAUTH_TOKEN="<token>"
|
||||
```
|
||||
|
||||
The Claude bottle reaches the Anthropic API only through the cred-proxy
|
||||
sidecar. To let `claude` authenticate, declare an egress route with
|
||||
`role: claude_code_oauth` and
|
||||
`token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||
|
||||
```yaml
|
||||
egress:
|
||||
routes:
|
||||
- host: api.anthropic.com
|
||||
role: claude_code_oauth
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
pipelock:
|
||||
tls_passthrough: true
|
||||
```
|
||||
|
||||
Routes that resolve to private or Tailscale addresses can opt into
|
||||
pipelock's SSRF destination allowlist explicitly:
|
||||
|
||||
```yaml
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
auth:
|
||||
scheme: token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
pipelock:
|
||||
ssrf_ip_allowlist:
|
||||
- 100.78.141.42/32
|
||||
```
|
||||
|
||||
At launch, `cli.py` reads `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` from the host
|
||||
env and forwards it into the cred-proxy container's environ — never
|
||||
into the agent's. The agent receives `ANTHROPIC_BASE_URL` pointing at
|
||||
`http://cred-proxy:9099/anthropic` and a non-secret placeholder for
|
||||
`CLAUDE_CODE_OAUTH_TOKEN` (claude-code refuses to start without one;
|
||||
the proxy strips and replaces the header on every request). `printenv`
|
||||
inside the agent does not surface the real token, and the value is
|
||||
never written to disk or placed on argv on the host.
|
||||
|
||||
A Claude bottle without a `claude_code_oauth` route has no path to the
|
||||
Anthropic API — there is no fallback that forwards the token directly
|
||||
to the agent. Caveats: the token is bound to your subscription tier
|
||||
(Pro/Max/Team/Enterprise), it does not work with `claude --bare`
|
||||
(which only reads `ANTHROPIC_API_KEY`), and if it leaks, regenerate
|
||||
via `claude setup-token` again. Reference:
|
||||
<https://code.claude.com/docs/en/authentication>.
|
||||
|
||||
## Trademarks
|
||||
|
||||
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
|
||||
bot-bottle is an independent project and is not affiliated with,
|
||||
endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude
|
||||
Code" are trademarks of Anthropic, PBC; the project name uses
|
||||
"claude" descriptively to indicate that the tool runs Claude Code
|
||||
inside a sandbox.
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2026 Eric Bauerfeld. Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text.
|
||||
Copyright 2026 Eric Bauerfeld
|
||||
|
||||
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE)
|
||||
for the full text.
|
||||
|
||||
@@ -4,15 +4,14 @@
|
||||
"env": {
|
||||
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
||||
},
|
||||
"git-gate": {
|
||||
"repos": {
|
||||
"foo": {
|
||||
"url": "ssh://git@upstream.invalid/path.git",
|
||||
"identity": "~/.cache/bot-bottle-demo/fake-key",
|
||||
"host_key": "ssh-ed25519 AAAAEXAMPLE"
|
||||
}
|
||||
"git": [
|
||||
{
|
||||
"Name": "foo",
|
||||
"Upstream": "ssh://git@upstream.invalid/path.git",
|
||||
"IdentityFile": "~/.cache/bot-bottle-demo/fake-key",
|
||||
"KnownHostKey": "ssh-ed25519 AAAAEXAMPLE"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
+32
-185
@@ -3,42 +3,18 @@
|
||||
The manifest owns the user-facing AgentProvider shape. This module is
|
||||
the launch-time table that turns a provider template into an executable
|
||||
command, default image, and prompt/auth behavior.
|
||||
|
||||
Per PRD 0050 the per-provider implementations live under
|
||||
`bot_bottle/contrib/<template>/agent_provider.py`. This module exposes:
|
||||
|
||||
- `AgentProvider` (ABC) — the contract each plugin implements.
|
||||
- `get_provider(template)` — lazy-imported registry; the analogue
|
||||
of `bot_bottle/deploy_key_provisioner.get_provisioner`.
|
||||
- `AgentProvisionPlan` (+ helper dataclasses) — declarative shape
|
||||
each provider produces and the backends consume unchanged.
|
||||
- `agent_provision_plan` / `runtime_for` — thin wrappers around the
|
||||
registry kept so existing callers keep working without per-call
|
||||
edits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from .egress import EgressRoute
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .backend import Bottle, BottlePlan
|
||||
from typing import Literal
|
||||
|
||||
|
||||
PROVIDER_CLAUDE = "claude"
|
||||
PROVIDER_CODEX = "codex"
|
||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
|
||||
|
||||
# Hosts that egress injects the host ChatGPT bearer on when Codex
|
||||
# forward_host_credentials is enabled. Pipelock must pass these through
|
||||
# (no TLS MITM) or its header DLP blocks the injected JWT.
|
||||
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
|
||||
PromptMode = Literal["append_file", "read_prompt_file"]
|
||||
|
||||
|
||||
@@ -48,176 +24,47 @@ class AgentProviderRuntime:
|
||||
command: str
|
||||
image: str
|
||||
dockerfile: str
|
||||
auth_role: str
|
||||
placeholder_env: str
|
||||
prompt_mode: PromptMode
|
||||
bypass_args: tuple[str, ...]
|
||||
resume_args: tuple[str, ...]
|
||||
remote_control_args: tuple[str, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionDir:
|
||||
guest_path: str
|
||||
mode: str = "700"
|
||||
owner: str = "node:node"
|
||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionFile:
|
||||
host_path: Path
|
||||
guest_path: str
|
||||
mode: str = "600"
|
||||
owner: str = "node:node"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionCommand:
|
||||
argv: tuple[str, ...]
|
||||
error: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionPlan:
|
||||
"""Provider-owned guest setup.
|
||||
|
||||
Backends interpret this plan with their own copy/exec primitives.
|
||||
Provider-specific content stays here so future provider plugins can
|
||||
return the same shape without adding backend-plan fields.
|
||||
|
||||
`egress_routes` are provider-declared EgressRoutes that backends
|
||||
pass to `Egress.prepare` and `PipelockProxy.prepare`. This keeps
|
||||
provider logic out of the egress and pipelock modules — they merge
|
||||
provider routes generically without knowing the provider type.
|
||||
|
||||
`hidden_env_names` is the set of env var names the provider injected
|
||||
as non-secret placeholders. `print_util.visible_agent_env_names` uses
|
||||
this to suppress them from the preflight summary so operators don't
|
||||
mistake them for real credentials.
|
||||
"""
|
||||
|
||||
template: str
|
||||
command: str
|
||||
prompt_mode: PromptMode
|
||||
image: str
|
||||
dockerfile: str
|
||||
guest_env: dict[str, str]
|
||||
env_vars: dict[str, str] = field(default_factory=dict)
|
||||
dirs: tuple[AgentProvisionDir, ...] = ()
|
||||
files: tuple[AgentProvisionFile, ...] = ()
|
||||
pre_copy: tuple[AgentProvisionCommand, ...] = ()
|
||||
verify: tuple[AgentProvisionCommand, ...] = ()
|
||||
egress_routes: tuple[EgressRoute, ...] = ()
|
||||
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
|
||||
provisioned_env: dict[str, str] = field(default_factory=dict)
|
||||
|
||||
|
||||
class AgentProvider(ABC):
|
||||
"""Per-template plugin: produces the provision plan and applies
|
||||
the provider-specific in-guest setup steps (skills, prompt, the
|
||||
declarative `dirs`/`files`/`pre_copy`/`verify` apply loop, and
|
||||
supervise MCP registration). Concrete subclasses live under
|
||||
`bot_bottle/contrib/<template>/agent_provider.py`."""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
"""The static command / image / prompt-mode table for this
|
||||
template."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
) -> AgentProvisionPlan:
|
||||
"""Build the declarative AgentProvisionPlan for one launch.
|
||||
Backends call this during `prepare` and consume the result as
|
||||
before."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each of the agent's named skills from the host into
|
||||
the guest. No-op when the agent has no skills. The in-guest
|
||||
layout is provider-specific (claude-code's
|
||||
`~/.claude/skills/` today; future providers may differ)."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode,
|
||||
and return the in-guest path iff the agent has a non-empty
|
||||
prompt (drives the `--append-system-prompt-file` flag).
|
||||
|
||||
The file is copied either way so the path always exists."""
|
||||
|
||||
@abstractmethod
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the provider's declarative
|
||||
`dirs`/`pre_copy`/`files`/`verify` steps from
|
||||
`plan.agent_provision`. Was called `provision_provider_auth`
|
||||
on `BottleBackend` before PRD 0050."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Register the per-bottle supervise sidecar as an MCP server
|
||||
in the provider's in-guest config. Called by the backend after
|
||||
the supervise sidecar is reachable. No-op when
|
||||
`plan.supervise_plan is None`."""
|
||||
|
||||
|
||||
def get_provider(template: str) -> AgentProvider:
|
||||
"""Resolve a provider template name to its plugin instance.
|
||||
|
||||
Lazy-imports the contrib module so importing this module doesn't
|
||||
pull provider-specific code paths in. Mirrors the contrib
|
||||
convention PRD 0048 established for deploy key provisioners."""
|
||||
if template == PROVIDER_CLAUDE:
|
||||
from .contrib.claude.agent_provider import ClaudeAgentProvider
|
||||
return ClaudeAgentProvider()
|
||||
if template == PROVIDER_CODEX:
|
||||
from .contrib.codex.agent_provider import CodexAgentProvider
|
||||
return CodexAgentProvider()
|
||||
raise ValueError(f"unknown agent provider template: {template!r}")
|
||||
_RUNTIMES = {
|
||||
PROVIDER_CLAUDE: AgentProviderRuntime(
|
||||
template=PROVIDER_CLAUDE,
|
||||
command="claude",
|
||||
image="bot-bottle-claude:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||
auth_role="claude_code_oauth",
|
||||
placeholder_env="CLAUDE_CODE_OAUTH_TOKEN",
|
||||
prompt_mode="append_file",
|
||||
bypass_args=("--dangerously-skip-permissions",),
|
||||
resume_args=("--continue",),
|
||||
remote_control_args=("--remote-control",),
|
||||
),
|
||||
PROVIDER_CODEX: AgentProviderRuntime(
|
||||
template=PROVIDER_CODEX,
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||
auth_role="codex_auth",
|
||||
placeholder_env="OPENAI_API_KEY",
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
resume_args=("resume", "--last"),
|
||||
remote_control_args=(),
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def runtime_for(template: str) -> AgentProviderRuntime:
|
||||
return get_provider(template).runtime
|
||||
|
||||
|
||||
def agent_provision_plan(
|
||||
*,
|
||||
template: str,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
) -> AgentProvisionPlan:
|
||||
"""Back-compat shim — `prepare` callers stay the same; the work
|
||||
now lives on the provider plugin."""
|
||||
return get_provider(template).provision_plan(
|
||||
dockerfile=dockerfile,
|
||||
state_dir=state_dir,
|
||||
guest_home=guest_home,
|
||||
guest_env=guest_env,
|
||||
auth_token=auth_token,
|
||||
forward_host_credentials=forward_host_credentials,
|
||||
host_env=host_env,
|
||||
trusted_project_path=trusted_project_path,
|
||||
)
|
||||
return _RUNTIMES[template]
|
||||
|
||||
|
||||
def prompt_args(
|
||||
|
||||
+43
-105
@@ -32,22 +32,15 @@ manifest does not carry a backend field; the host picks.
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from abc import ABC, abstractmethod
|
||||
from contextlib import AbstractContextManager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Generic, Sequence, TypeVar
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan, get_provider
|
||||
from ..egress import EgressPlan
|
||||
from ..git_gate import GitGatePlan
|
||||
from ..log import die, info
|
||||
from ..log import die
|
||||
from ..manifest import GitEntry, Manifest
|
||||
from ..supervise import SupervisePlan
|
||||
from ..util import expand_tilde
|
||||
from ..workspace import WorkspacePlan
|
||||
from .print_util import print_multi, visible_agent_env_names
|
||||
from .util import host_skill_dir
|
||||
|
||||
|
||||
@@ -72,58 +65,15 @@ class BottleSpec:
|
||||
@dataclass(frozen=True)
|
||||
class BottlePlan(ABC):
|
||||
"""Base output of a backend's prepare step. Concrete subclasses
|
||||
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
|
||||
(e.g. DockerBottlePlan) add backend-specific resolved fields and
|
||||
implement `print`."""
|
||||
|
||||
spec: BottleSpec
|
||||
stage_dir: Path
|
||||
guest_home: str
|
||||
git_gate_plan: GitGatePlan
|
||||
egress_plan: EgressPlan
|
||||
supervise_plan: SupervisePlan | None
|
||||
agent_provision: AgentProvisionPlan
|
||||
workspace_plan: WorkspacePlan
|
||||
|
||||
@abstractmethod
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Render the y/N preflight summary to stderr."""
|
||||
del remote_control
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(
|
||||
set(bottle.env.keys())
|
||||
| set(self.agent_provision.guest_env.keys())
|
||||
),
|
||||
hidden_env_names=self.agent_provision.hidden_env_names,
|
||||
)
|
||||
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provision.template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
|
||||
identity = manifest.git_identity_summary(spec.agent_name)
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
|
||||
git_lines = [
|
||||
f"{u.name} → {u.upstream_host}:{u.upstream_port}"
|
||||
for u in self.git_gate_plan.upstreams
|
||||
]
|
||||
if git_lines:
|
||||
print_multi(" git gate ", git_lines)
|
||||
|
||||
if self.egress_plan.routes:
|
||||
egress_lines = []
|
||||
for r in self.egress_plan.routes:
|
||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||
egress_lines.append(f"{r.host}{auth}")
|
||||
print_multi(" egress ", egress_lines)
|
||||
print(file=sys.stderr)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -313,44 +263,35 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||
"""Build/run the bottle and yield a handle; tear down on exit."""
|
||||
|
||||
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
|
||||
def provision(self, plan: PlanT, target: str) -> str | None:
|
||||
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
||||
the running bottle. Called from `launch` after the container
|
||||
/ machine is up. Returns the in-container prompt path if a
|
||||
prompt was provisioned, else None — the Bottle handle uses it
|
||||
to decide whether to add provider-specific prompt args to the
|
||||
agent's argv.
|
||||
/ machine is up. `target` identifies the running instance in
|
||||
backend-specific terms (Docker: resolved container name; fly:
|
||||
machine id). Returns the in-container prompt path if a prompt
|
||||
was provisioned, else None — the Bottle handle uses it to
|
||||
decide whether to add provider-specific prompt args to the agent's
|
||||
argv.
|
||||
|
||||
Default orchestration: ca → prompt → provider apply → skills
|
||||
→ workspace → git → supervise-mcp. CA install runs first so
|
||||
the agent's trust store is rebuilt before anything inside the
|
||||
agent makes a TLS call.
|
||||
|
||||
Per PRD 0050 the per-provider steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin. The backend only owns the
|
||||
steps that are about backend infrastructure (CA, workspace,
|
||||
git) and surfaces the supervise sidecar URL its launch step
|
||||
knows about via `supervise_mcp_url`.
|
||||
Default orchestration: ca → prompt → skills → git →
|
||||
supervise. CA install runs first so the agent's trust store
|
||||
is rebuilt before anything inside the agent makes a TLS call.
|
||||
Subclasses typically don't override this; they implement the
|
||||
sub-methods below.
|
||||
|
||||
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
||||
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
||||
on the agent's HTTP_PROXY path so every tool that respects
|
||||
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
||||
intercepted without per-tool reconfiguration."""
|
||||
provider = get_provider(plan.agent_provision.template)
|
||||
self.provision_ca(plan, bottle)
|
||||
prompt_path = provider.provision_prompt(plan, bottle)
|
||||
provider.provision(plan, bottle)
|
||||
provider.provision_skills(plan, bottle)
|
||||
self.provision_workspace(plan, bottle)
|
||||
self.provision_git(plan, bottle)
|
||||
provider.provision_supervise_mcp(
|
||||
plan, bottle, self.supervise_mcp_url(plan),
|
||||
)
|
||||
self.provision_ca(plan, target)
|
||||
prompt_path = self.provision_prompt(plan, target)
|
||||
self.provision_skills(plan, target)
|
||||
self.provision_git(plan, target)
|
||||
self.provision_supervise(plan, target)
|
||||
return prompt_path
|
||||
|
||||
def provision_ca(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||
def provision_ca(self, plan: PlanT, target: str) -> None:
|
||||
"""Install the per-bottle CA into the agent's trust store so
|
||||
the agent trusts the bumped CONNECT cert egress (was
|
||||
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
|
||||
@@ -359,26 +300,29 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
backend overrides to docker-cp the cert in and run
|
||||
`update-ca-certificates`."""
|
||||
|
||||
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||
"""Copy the operator workspace into the running bottle when
|
||||
the backend cannot bake it into the agent image. Default is
|
||||
no-op for backends like Docker that handle this before launch."""
|
||||
@abstractmethod
|
||||
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
||||
"""Copy the prompt file into the running bottle. Returns the
|
||||
in-container path iff the agent has a non-empty prompt;
|
||||
callers use the return value to decide whether to add
|
||||
provider-specific prompt args to the agent's argv."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_git(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||
def provision_skills(self, plan: PlanT, target: str) -> None:
|
||||
"""Copy the agent's named skills from the host into the
|
||||
running bottle. No-op when the agent has no skills."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_git(self, plan: PlanT, target: str) -> None:
|
||||
"""Copy the host's cwd `.git` directory into the running
|
||||
bottle if the user requested --cwd. No-op otherwise."""
|
||||
|
||||
def supervise_mcp_url(self, plan: PlanT) -> str:
|
||||
"""Return the agent-side URL of the per-bottle supervise
|
||||
sidecar, or "" when this bottle has no sidecar. The provider
|
||||
plugin's `provision_supervise_mcp` uses it to register the
|
||||
MCP entry inside the guest.
|
||||
|
||||
Default returns "" so backends without supervise support
|
||||
don't have to implement it. Docker and smolmachines override."""
|
||||
del plan
|
||||
return ""
|
||||
def provision_supervise(self, plan: PlanT, target: str) -> None:
|
||||
"""Write the in-bottle Claude Code MCP config so the agent
|
||||
discovers the per-bottle supervise sidecar (PRD 0013).
|
||||
No-op when bottle.supervise is False or the backend doesn't
|
||||
support the supervise sidecar yet. The Docker backend
|
||||
overrides."""
|
||||
|
||||
@abstractmethod
|
||||
def prepare_cleanup(self) -> CleanupT:
|
||||
@@ -469,20 +413,14 @@ def enumerate_active_agents() -> list[ActiveAgent]:
|
||||
"""All currently-running agents, across every available
|
||||
backend. Used by CLI `list active` and the dashboard's agents
|
||||
pane so neither has to know which backends exist. Skips
|
||||
backends whose `is_available()` reports False.
|
||||
|
||||
Sorted by `(started_at, slug)` so the list is stable across
|
||||
dashboard refresh ticks — agents don't shift position while
|
||||
the operator navigates with arrow keys. ISO 8601 timestamps
|
||||
sort lexicographically in chronological order; `slug` is the
|
||||
deterministic tiebreaker. Agents with missing metadata
|
||||
(`started_at == ""`) sort first."""
|
||||
backends whose `is_available()` reports False. Ordered by
|
||||
backend name, then by whatever each backend's
|
||||
`enumerate_active` returns."""
|
||||
out: list[ActiveAgent] = []
|
||||
for name in known_backend_names():
|
||||
if not has_backend(name):
|
||||
continue
|
||||
out.extend(_BACKENDS[name].enumerate_active())
|
||||
out.sort(key=lambda a: (a.started_at, a.slug))
|
||||
return out
|
||||
|
||||
|
||||
|
||||
@@ -9,12 +9,6 @@ This module is a thin façade. The real work lives in four siblings:
|
||||
|
||||
The base class's `prepare` template runs cross-backend host-side
|
||||
validation before calling `_resolve_plan` here.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
the declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||
Docker backend only owns the steps that are about backend
|
||||
infrastructure: CA install and git copy-in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -24,8 +18,7 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
@@ -35,6 +28,9 @@ from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .provision import ca as _ca
|
||||
from .provision import git as _git
|
||||
from .provision import prompt as _prompt
|
||||
from .provision import skills as _skills
|
||||
from .provision import supervise as _supervise_prov
|
||||
|
||||
|
||||
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
||||
@@ -60,19 +56,20 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def provision_ca(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
_ca.provision_ca(plan, bottle)
|
||||
def provision_ca(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_ca.provision_ca(plan, target)
|
||||
|
||||
def provision_git(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
_git.provision_git(plan, bottle)
|
||||
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
||||
return _prompt.provision_prompt(plan, target)
|
||||
|
||||
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
||||
"""Docker bottles reach the supervise sidecar via the
|
||||
compose-network alias `supervise:9100`. No per-bottle URL
|
||||
plumbing needed; the alias resolves inside the bridge."""
|
||||
if plan.supervise_plan is None:
|
||||
return ""
|
||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_skills.provision_skills(plan, target)
|
||||
|
||||
def provision_git(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_git.provision_git(plan, target)
|
||||
|
||||
def provision_supervise(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_supervise_prov.provision_supervise(plan, target)
|
||||
|
||||
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
@@ -2,25 +2,30 @@
|
||||
|
||||
Carries the Docker-specific resolved fields produced by
|
||||
DockerBottleBackend.prepare. The launch step consumes it without
|
||||
further resolution; preflight rendering is inherited from BottlePlan.
|
||||
further resolution; show_plan-style rendering is the `print` method.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from ...egress import EgressPlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...log import info
|
||||
from ...pipelock import PipelockProxyPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottlePlan
|
||||
from ..print_util import print_multi, visible_agent_env_names
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DockerBottlePlan(BottlePlan):
|
||||
"""Docker-specific resolved fields produced by
|
||||
DockerBottleBackend.prepare. Inherits `spec`, `stage_dir`,
|
||||
`git_gate_plan`, `egress_plan`, `supervise_plan`, and
|
||||
`agent_provision` from BottlePlan."""
|
||||
DockerBottleBackend.prepare. Inherits `spec` and `stage_dir` from
|
||||
BottlePlan."""
|
||||
|
||||
slug: str
|
||||
container_name: str
|
||||
@@ -41,16 +46,56 @@ class DockerBottlePlan(BottlePlan):
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
prompt_file: Path
|
||||
proxy_plan: PipelockProxyPlan
|
||||
git_gate_plan: GitGatePlan
|
||||
egress_plan: EgressPlan
|
||||
# None when bottle.supervise is False. PRD 0013 supervise sidecar
|
||||
# is opt-in via the manifest's bottle.supervise field.
|
||||
supervise_plan: SupervisePlan | None
|
||||
use_runsc: bool
|
||||
agent_command: str = "claude"
|
||||
agent_prompt_mode: PromptMode = "append_file"
|
||||
agent_provider_template: str = "claude"
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Render the y/N preflight summary to stderr — compact form
|
||||
intended to fit on screen without scrolling. The full
|
||||
structured shape (image, container, runtime, etc.) lives on
|
||||
this dataclass for tooling that wants to introspect it."""
|
||||
del remote_control # not surfaced in the compact summary
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
# The agent sees the union of literal env names (rendered into
|
||||
# --env-file) and forwarded env names (`-e NAME` with the
|
||||
# value arriving via subprocess env). The forwarded set holds
|
||||
# the OAuth token (CLAUDE_CODE_OAUTH_TOKEN) and any host-env
|
||||
# interpolations from the manifest; egress holds
|
||||
# upstream tokens in its own environ, so no token forwarding
|
||||
# from the agent to the proxy is needed.
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(set(bottle.env.keys()) | set(self.forwarded_env.keys())),
|
||||
agent_provider_template=self.agent_provider_template,
|
||||
)
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provider_template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
git_lines = [
|
||||
f"{u.upstream_host}:{u.upstream_port}"
|
||||
for u in self.git_gate_plan.upstreams
|
||||
]
|
||||
if git_lines:
|
||||
print_multi(" git gate ", git_lines)
|
||||
|
||||
if self.egress_plan.routes:
|
||||
egress_lines = []
|
||||
for r in self.egress_plan.routes:
|
||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||
egress_lines.append(f"{r.host}{auth}")
|
||||
print_multi(" egress ", egress_lines)
|
||||
print(file=sys.stderr)
|
||||
|
||||
@@ -105,10 +105,6 @@ class BottleMetadata:
|
||||
# written before chunk 3 (resume / inspect should fall back to
|
||||
# deriving from identity in that case).
|
||||
compose_project: str = ""
|
||||
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
|
||||
# for state dirs written before PRD 0040; callers default to "docker"
|
||||
# for backward compatibility.
|
||||
backend: str = ""
|
||||
|
||||
|
||||
def metadata_path(identity: str) -> Path:
|
||||
@@ -142,7 +138,6 @@ def read_metadata(identity: str) -> BottleMetadata | None:
|
||||
copy_cwd=bool(raw.get("copy_cwd", False)),
|
||||
started_at=str(raw.get("started_at", "")),
|
||||
compose_project=str(raw.get("compose_project", "")),
|
||||
backend=str(raw.get("backend", "")),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ from ...egress import (
|
||||
EGRESS_HOSTNAME,
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
)
|
||||
from ...git_gate import GIT_GATE_HOSTNAME
|
||||
from ...git_gate import GIT_GATE_HOSTNAME, git_gate_aggregate_extra_hosts
|
||||
from ...log import die, warn
|
||||
from ...pipelock import PIPELOCK_HOSTNAME
|
||||
from ...supervise import (
|
||||
@@ -198,6 +198,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
env.append(token_env)
|
||||
|
||||
# --- git-gate ----------------------------------------------------
|
||||
extra_hosts: list[str] = []
|
||||
gp = plan.git_gate_plan
|
||||
if gp.upstreams:
|
||||
volumes += [
|
||||
@@ -216,6 +217,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
u.known_hosts_file,
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||
))
|
||||
extra_map = git_gate_aggregate_extra_hosts(gp.upstreams)
|
||||
extra_hosts = [f"{host}:{ip}" for host, ip in sorted(extra_map.items())]
|
||||
|
||||
# --- supervise ---------------------------------------------------
|
||||
sp = plan.supervise_plan
|
||||
@@ -258,6 +261,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"environment": env,
|
||||
"volumes": volumes,
|
||||
}
|
||||
if extra_hosts:
|
||||
service["extra_hosts"] = extra_hosts
|
||||
return service
|
||||
|
||||
|
||||
@@ -281,8 +286,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
||||
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
||||
]
|
||||
for name, value in sorted(plan.agent_provision.guest_env.items()):
|
||||
env.append(f"{name}={value}")
|
||||
# Forwarded vars (OAuth token, manifest host-interpolations):
|
||||
# bare name → inherits from compose-up process env, value
|
||||
# never lands on argv or in the compose file.
|
||||
|
||||
@@ -43,8 +43,7 @@ from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import egress_resolve_token_values
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import info, warn
|
||||
from ...log import info
|
||||
from . import network as network_mod
|
||||
from . import util as docker_mod
|
||||
from .bottle import DockerBottle
|
||||
@@ -52,7 +51,6 @@ from .bottle_plan import DockerBottlePlan
|
||||
from .bottle_state import (
|
||||
bottle_state_dir,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
pipelock_state_dir,
|
||||
)
|
||||
from .compose import (
|
||||
@@ -86,20 +84,13 @@ def launch(
|
||||
Teardown on exit."""
|
||||
stack = ExitStack()
|
||||
|
||||
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc:
|
||||
warn(
|
||||
f"teardown failed for container {plan.container_name}"
|
||||
f" (compose-down): {exc!r}"
|
||||
)
|
||||
revoke_git_gate_provisioned_keys(
|
||||
_bottle_for_revoke, _git_gate_dir_for_revoke
|
||||
)
|
||||
except BaseException:
|
||||
# Teardown must not raise; swallow so the caller's
|
||||
# __exit__ path can still propagate the original error.
|
||||
pass
|
||||
|
||||
try:
|
||||
# Step 1: agent image build. Sidecar images get built lazily by
|
||||
@@ -110,7 +101,7 @@ def launch(
|
||||
)
|
||||
if plan.derived_image:
|
||||
docker_mod.build_image_with_cwd(
|
||||
plan.derived_image, plan.image, plan.workspace_plan
|
||||
plan.derived_image, plan.image, plan.spec.user_cwd
|
||||
)
|
||||
|
||||
# Networks: compose-managed. The names are derived
|
||||
@@ -185,10 +176,11 @@ def launch(
|
||||
# Step 7: compose up. Token values + the OAuth placeholder
|
||||
# flow through subprocess env; the compose file holds only
|
||||
# bare names for the secret-carrying entries.
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, effective_env,
|
||||
)
|
||||
token_values: dict[str, str] = {}
|
||||
if plan.egress_plan.routes:
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, dict(os.environ),
|
||||
)
|
||||
compose_env: dict[str, str] = {
|
||||
**os.environ,
|
||||
**plan.forwarded_env,
|
||||
@@ -208,21 +200,19 @@ def launch(
|
||||
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
||||
)
|
||||
|
||||
# Step 8: provision. Create the bottle first so provisioners
|
||||
# can use bottle.exec / bottle.cp_in; set the prompt path
|
||||
# returned by provision_prompt after the fact.
|
||||
bottle = DockerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
None,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
)
|
||||
bottle._prompt_path = provision(plan, bottle)
|
||||
# Step 8: provision. Unchanged — uses `docker exec` against
|
||||
# the agent container by its known name.
|
||||
prompt_path = provision(plan, plan.container_name)
|
||||
|
||||
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
||||
# — the agent runs `sleep infinity` per the renderer's
|
||||
# service spec.
|
||||
yield bottle
|
||||
yield DockerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
prompt_path,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
)
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
@@ -12,17 +12,15 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import agent_provision_plan, runtime_for
|
||||
from ...agent_provider import runtime_for
|
||||
from ...egress import Egress
|
||||
from ...env import ResolvedEnv, resolve_env
|
||||
from ...git_gate import GitGate
|
||||
from ...log import die
|
||||
from ...pipelock import PipelockProxy
|
||||
from ...supervise import Supervise
|
||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||
from .. import BottleSpec
|
||||
from . import util as docker_mod
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
@@ -63,8 +61,6 @@ def resolve_plan(
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
guest_home = "/home/node"
|
||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||
|
||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||
# mints a random-suffixed identity (so parallel runs of the same
|
||||
@@ -82,7 +78,6 @@ def resolve_plan(
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project=f"bot-bottle-{slug}",
|
||||
backend="docker",
|
||||
))
|
||||
# Clear any leftover preserve marker from a prior capability-block
|
||||
# so this fresh launch can be cleaned up at session-end unless
|
||||
@@ -163,45 +158,17 @@ def resolve_plan(
|
||||
prompt_file.write_text("")
|
||||
prompt_file.chmod(0o600)
|
||||
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = proxy.prepare(bottle, slug, pipelock_dir)
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
# Everything that should reach the bottle by-name (so its value
|
||||
# never lands on argv or in env_file) goes into one dict. Nothing
|
||||
# mutates the host os.environ.
|
||||
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
||||
_write_env_file(resolved, env_file)
|
||||
prompt_file.write_text(agent.prompt)
|
||||
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
agent_provision = agent_provision_plan(
|
||||
template=provider.template,
|
||||
dockerfile=dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
guest_home=guest_home,
|
||||
forward_host_credentials=provider.forward_host_credentials,
|
||||
auth_token=provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace_plan.workdir,
|
||||
)
|
||||
guest_env = dict(agent_provision.guest_env)
|
||||
for key, val in agent_provision.env_vars.items():
|
||||
guest_env.setdefault(key, val)
|
||||
agent_provision = replace(agent_provision, guest_env=guest_env)
|
||||
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = proxy.prepare(
|
||||
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
||||
)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = egress.prepare(
|
||||
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||
)
|
||||
egress_plan = egress.prepare(bottle, slug, egress_dir)
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
@@ -229,11 +196,37 @@ def resolve_plan(
|
||||
slug, supervise_dir,
|
||||
dockerfile_content=dockerfile_content,
|
||||
)
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
# Everything that should reach the bottle by-name (so its value
|
||||
# never lands on argv or in env_file) goes into one dict. Nothing
|
||||
# mutates the host os.environ.
|
||||
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
||||
# When the bottle declares an egress route with the
|
||||
# `claude_code_oauth` role marker, claude-code's outbound
|
||||
# Authorization gets stripped + re-injected by egress. The
|
||||
# agent's environ still needs *something* claude-code recognises
|
||||
# as a credential or it refuses to start; ship a non-secret
|
||||
# placeholder. The placeholder isn't any real token value, so
|
||||
# leaking it would tell an attacker only that egress is in
|
||||
# front. Manifest validation enforces singleton on this role.
|
||||
has_provider_auth = any(
|
||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
||||
)
|
||||
if has_provider_auth:
|
||||
forwarded_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
||||
if provider.template == "claude" and has_provider_auth:
|
||||
# Belt-and-braces: turn off telemetry endpoints (statsig,
|
||||
# error reporting) that egress can't gate by auth.
|
||||
forwarded_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
||||
forwarded_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
||||
_write_env_file(resolved, env_file)
|
||||
prompt_file.write_text(agent.prompt)
|
||||
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
guest_home=guest_home,
|
||||
slug=slug,
|
||||
container_name=container_name,
|
||||
container_name_pinned=container_name_pinned,
|
||||
@@ -249,8 +242,9 @@ def resolve_plan(
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
use_runsc=use_runsc,
|
||||
agent_provision=agent_provision,
|
||||
workspace_plan=workspace_plan,
|
||||
agent_command=provider_runtime.command,
|
||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
||||
agent_provider_template=provider.template,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
"""Backend-infrastructure provisioners for the Docker backend.
|
||||
"""Per-provisioner modules for the Docker backend.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration) live on
|
||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
|
||||
left in this subpackage handle only the steps that are
|
||||
backend-specific:
|
||||
Each module exports one top-level function:
|
||||
provision_<thing>(plan: DockerBottlePlan, target: str) -> ...
|
||||
|
||||
- ca.py — install per-bottle CA bundle into the guest trust store
|
||||
- git.py — copy host cwd `.git` into the guest when --cwd is used
|
||||
"""
|
||||
`DockerBottleBackend.provision_*` methods delegate to these. The
|
||||
abstract `BottleBackend.provision_*` surface is unchanged; this
|
||||
subpackage exists only to keep `backend.py` from being a god-file."""
|
||||
|
||||
@@ -31,21 +31,33 @@ stage dir; nothing in the agent ever sees it."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ... import Bottle
|
||||
import subprocess
|
||||
|
||||
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_ca(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
def provision_ca(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Copy the agent-facing CA cert into the agent, rebuild the
|
||||
trust bundle, emit a one-line fingerprint log. Called from
|
||||
`BottleBackend.provision` after the agent container is up."""
|
||||
container = target
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||
|
||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||
bottle.exec(
|
||||
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
||||
user="root",
|
||||
subprocess.run(
|
||||
["docker", "cp", str(cert_host_path), f"{container}:{AGENT_CA_PATH}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chmod", "644", AGENT_CA_PATH],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "update-ca-certificates"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
log_ca_fingerprint(cert_host_path, label)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
Three concerns, all about git in the agent:
|
||||
|
||||
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
||||
into the planned guest workspace so the agent operates on the
|
||||
into /home/node/workspace/.git so the agent operates on the
|
||||
user's repo.
|
||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||
~/.gitconfig with insteadOf rules so every git operation
|
||||
@@ -18,62 +18,73 @@ Three concerns, all about git in the agent:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||
from ....log import info
|
||||
from ... import Bottle
|
||||
from .. import util as docker_mod
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
def provision_git(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Set up git inside the bottle. Runs all three subcases; each
|
||||
no-ops when its condition isn't met."""
|
||||
_provision_cwd_git(plan, bottle)
|
||||
_provision_git_gate_config(plan, bottle)
|
||||
_provision_git_user(plan, bottle)
|
||||
_provision_cwd_git(plan, target)
|
||||
_provision_git_gate_config(plan, target)
|
||||
_provision_git_user(plan, target)
|
||||
|
||||
|
||||
def _provision_cwd_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||
it into /home/node/workspace/.git and fix ownership. No-op
|
||||
otherwise."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||
return
|
||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||
host_git = str(workspace.host_path / ".git")
|
||||
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
||||
bottle.cp_in(host_git, guest_workspace_git)
|
||||
bottle.exec(
|
||||
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
|
||||
user="root",
|
||||
container = target
|
||||
info(f"copying {plan.spec.user_cwd}/.git -> {container}:/home/node/workspace/.git")
|
||||
subprocess.run(
|
||||
["docker", "cp", f"{plan.spec.user_cwd}/.git", f"{container}:/home/node/workspace/.git"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
[
|
||||
"docker", "exec", "-u", "0", container,
|
||||
"chown", "-R", "node:node", "/home/node/workspace/.git",
|
||||
],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def _provision_git_gate_config(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
def _provision_git_gate_config(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Write ~/.gitconfig in the bottle with the git-gate
|
||||
insteadOf rules. No-op when the bottle has no `git` entries."""
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not manifest_bottle.git:
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not bottle.git:
|
||||
return
|
||||
container_gitconfig = f"{plan.guest_home}/.gitconfig"
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
container_gitconfig = f"{container_home}/.gitconfig"
|
||||
|
||||
content = git_gate_render_gitconfig(manifest_bottle.git, GIT_GATE_HOSTNAME)
|
||||
content = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
||||
config_file = plan.stage_dir / "agent_gitconfig"
|
||||
config_file.write_text(content)
|
||||
config_file.chmod(0o600)
|
||||
|
||||
info(f"writing {container_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
|
||||
bottle.cp_in(str(config_file), container_gitconfig)
|
||||
bottle.exec(
|
||||
f"chown node:node {shlex.quote(container_gitconfig)} && "
|
||||
f"chmod 644 {shlex.quote(container_gitconfig)}",
|
||||
user="root",
|
||||
info(f"writing {container_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||
subprocess.run(
|
||||
["docker", "cp", str(config_file), f"{container}:{container_gitconfig}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
docker_mod.docker_exec_root(container, ["chown", "node:node", container_gitconfig])
|
||||
docker_mod.docker_exec_root(container, ["chmod", "644", container_gitconfig])
|
||||
|
||||
|
||||
def _provision_git_user(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
def _provision_git_user(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Apply `git config --global user.{name,email}` inside the
|
||||
bottle so the agent's commits are attributed to the operator-
|
||||
chosen identity instead of the agent image's default
|
||||
@@ -88,19 +99,23 @@ def _provision_git_user(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
||||
Each field set independently — name-only or email-only
|
||||
configs only run the `git config` line for the field
|
||||
present."""
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = manifest_bottle.git_user
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = bottle.git_user
|
||||
if gu.is_empty():
|
||||
return
|
||||
if gu.name:
|
||||
info(f"git config --global user.name = {gu.name!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.name {shlex.quote(gu.name)}",
|
||||
user="node",
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "node", target,
|
||||
"git", "config", "--global", "user.name", gu.name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
if gu.email:
|
||||
info(f"git config --global user.email = {gu.email!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.email {shlex.quote(gu.email)}",
|
||||
user="node",
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "node", target,
|
||||
"git", "config", "--global", "user.email", gu.email],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Copy the agent prompt into a running Docker bottle.
|
||||
|
||||
The prompt file is always copied (so the in-container path always
|
||||
exists) but `--append-system-prompt-file` only fires when the agent
|
||||
actually has a prompt — the return value signals which case."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_prompt(plan: DockerBottlePlan, target: str) -> str | None:
|
||||
"""Copy the prompt file into the container, fix ownership/mode.
|
||||
Returns the in-container path if the agent has a non-empty
|
||||
prompt (drives --append-system-prompt-file), else None. The
|
||||
file is copied either way so the path always exists."""
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
in_container_prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
subprocess.run(
|
||||
["docker", "cp", str(plan.prompt_file), f"{container}:{in_container_prompt_path}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
# `docker cp` preserves host UID; re-own/mode as root so node
|
||||
# can read its own mode-600 prompt regardless of host UID.
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chown", "node:node", in_container_prompt_path],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chmod", "600", in_container_prompt_path],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return in_container_prompt_path if agent.prompt else None
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Copy host-side skill directories into a running Docker bottle.
|
||||
|
||||
Skills are validated on the host before launch by the base class's
|
||||
`BottleBackend._validate_skills` (called from `prepare`); this module
|
||||
assumes that validation has already run. A skill disappearing between
|
||||
validation and copy still dies loudly rather than silently producing
|
||||
a partial container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ....log import die, info
|
||||
from ...util import host_skill_dir
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_skills(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Copy each of the agent's named skills from the host's
|
||||
~/.claude/skills/<name>/ into the container's equivalent path.
|
||||
For each skill: ensure parent dir, wipe any prior copy, then
|
||||
`docker cp <host>/. <container>:<dst>/` so the contents are
|
||||
copied into a freshly-created destination dir. No-op when the
|
||||
agent has no skills."""
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
skills_dir = os.environ.get(
|
||||
"BOT_BOTTLE_CONTAINER_SKILLS_DIR", f"{container_home}/.claude/skills"
|
||||
)
|
||||
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "mkdir", "-p", skills_dir],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
for n in agent.skills:
|
||||
src = host_skill_dir(n)
|
||||
if not os.path.isdir(src):
|
||||
die(f"skill '{n}' disappeared from host between validation and copy at {src}.")
|
||||
dst = f"{skills_dir}/{n}"
|
||||
info(f"copying skill {n} into {container}:{dst}")
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "rm", "-rf", dst],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "mkdir", "-p", dst],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "cp", f"{src}/.", f"{container}:{dst}/"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Supervise sidecar provisioning inside a running Docker bottle
|
||||
(PRD 0013).
|
||||
|
||||
Registers the per-bottle supervise sidecar as an HTTP MCP server in
|
||||
the agent's claude-code config so the agent discovers the three
|
||||
stuck-recovery MCP tools (cred-proxy-block, pipelock-block,
|
||||
capability-block) at startup.
|
||||
|
||||
Uses `claude mcp add` rather than writing JSON directly. claude-code
|
||||
owns the on-disk config format (`~/.claude.json` `mcpServers` shape,
|
||||
field names, scope semantics) and changes it between versions; the
|
||||
official command handles whatever the installed version expects.
|
||||
|
||||
No-op when bottle.supervise is False — bottles that haven't opted
|
||||
into the supervise sidecar shouldn't get an MCP entry pointing at a
|
||||
sidecar that isn't running.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ....log import info, warn
|
||||
from ....supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def supervise_mcp_url() -> str:
|
||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||
|
||||
|
||||
def provision_supervise(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Run `claude mcp add` inside the agent container to register
|
||||
the supervise sidecar in claude-code's user config. No-op when
|
||||
bottle.supervise is False.
|
||||
|
||||
Failure is logged but not fatal: the bottle still works (you
|
||||
just can't call supervise tools from the agent until the entry
|
||||
is added manually). The operator sees the warning at launch."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
url = supervise_mcp_url()
|
||||
argv = [
|
||||
"docker", "exec", "-u", "node", target,
|
||||
"claude", "mcp", "add",
|
||||
"--scope", "user",
|
||||
"--transport", "http",
|
||||
_SUPERVISE_MCP_NAME,
|
||||
url,
|
||||
]
|
||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||
r = subprocess.run(argv, capture_output=True, text=True, check=False)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {url}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["provision_supervise", "supervise_mcp_url"]
|
||||
@@ -7,11 +7,9 @@ from __future__ import annotations
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from typing import Iterable, Iterator
|
||||
|
||||
from ...log import die, info
|
||||
from ...workspace import WorkspacePlan
|
||||
|
||||
|
||||
# Cap on the suffix the container-name conflict logic will try before
|
||||
@@ -118,39 +116,35 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
subprocess.run(args, check=True)
|
||||
|
||||
|
||||
def build_image_with_cwd(
|
||||
derived: str,
|
||||
base: str,
|
||||
workspace: WorkspacePlan,
|
||||
) -> None:
|
||||
"""Build a thin derived image that copies the workspace into
|
||||
the plan's guest path and sets the plan's workdir."""
|
||||
_TRUST_DIALOG_NODE_SCRIPT = (
|
||||
'const fs=require("fs"),p=process.env.HOME+"/.claude.json",'
|
||||
'c=JSON.parse(fs.readFileSync(p,"utf8"));'
|
||||
'c.projects=c.projects||{};'
|
||||
'c.projects[process.env.HOME+"/workspace"]={hasTrustDialogAccepted:true};'
|
||||
'fs.writeFileSync(p,JSON.stringify(c,null,2));'
|
||||
)
|
||||
|
||||
|
||||
def build_image_with_cwd(derived: str, base: str, cwd: str) -> None:
|
||||
"""Build a thin derived image that copies <cwd> into
|
||||
/home/node/workspace and adds a trust-dialog entry for it."""
|
||||
import os
|
||||
|
||||
cwd = str(workspace.host_path)
|
||||
if not os.path.isdir(cwd):
|
||||
die(f"cwd not found at {cwd}")
|
||||
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
||||
context_dir = os.path.join(tmp, "context")
|
||||
staged_workspace = os.path.join(context_dir, "workspace")
|
||||
shutil.copytree(
|
||||
cwd,
|
||||
staged_workspace,
|
||||
symlinks=True,
|
||||
ignore=shutil.ignore_patterns(".git"),
|
||||
)
|
||||
dockerfile = (
|
||||
f"FROM {base}\n"
|
||||
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
||||
f"WORKDIR {workspace.workdir}\n"
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "build", "-t", derived, "-f", "-", context_dir],
|
||||
input=dockerfile,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
info(f"building image {derived} from {base} with {cwd} -> /home/node/workspace")
|
||||
dockerfile = (
|
||||
f"FROM {base}\n"
|
||||
f"COPY --chown=node:node . /home/node/workspace\n"
|
||||
f"RUN node -e '{_TRUST_DIALOG_NODE_SCRIPT}'\n"
|
||||
f"WORKDIR /home/node/workspace\n"
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "build", "-t", derived, "-f", "-", cwd],
|
||||
input=dockerfile,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Sequence
|
||||
|
||||
from ..agent_provider import runtime_for
|
||||
from ..log import info
|
||||
|
||||
|
||||
@@ -29,13 +30,16 @@ def print_multi(label: str, values: Sequence[str]) -> None:
|
||||
|
||||
|
||||
def visible_agent_env_names(
|
||||
env_names: Sequence[str], *, hidden_env_names: frozenset[str],
|
||||
env_names: Sequence[str], *, agent_provider_template: str,
|
||||
) -> list[str]:
|
||||
"""Env names worth showing in launch summaries.
|
||||
|
||||
Provider-injected placeholder env vars are implementation details:
|
||||
they are non-secret dummy values that satisfy provider CLIs while
|
||||
egress injects the real Authorization header. The plan's
|
||||
`hidden_env_names` carries exactly which names to suppress.
|
||||
Provider auth placeholders (`OPENAI_API_KEY`,
|
||||
`CLAUDE_CODE_OAUTH_TOKEN`) are implementation details: they are
|
||||
non-secret dummy values that satisfy the provider CLI while egress
|
||||
injects the real upstream Authorization header. Showing them in
|
||||
preflight makes the operator think a real key is entering the
|
||||
agent, so hide only that provider-owned placeholder.
|
||||
"""
|
||||
return sorted({name for name in env_names if name and name not in hidden_env_names})
|
||||
hidden = {runtime_for(agent_provider_template).placeholder_env}
|
||||
return sorted({name for name in env_names if name not in hidden})
|
||||
|
||||
@@ -1,11 +1,5 @@
|
||||
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
||||
BottleBackend (PRD 0023).
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
the declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||
smolmachines backend only owns the steps that are about backend
|
||||
infrastructure: CA install (no-op for now), workspace, git copy-in."""
|
||||
BottleBackend (PRD 0023)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -13,7 +7,7 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
@@ -24,7 +18,9 @@ from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .provision import ca as _ca
|
||||
from .provision import git as _git
|
||||
from .provision import workspace as _workspace
|
||||
from .provision import prompt as _prompt
|
||||
from .provision import skills as _skills
|
||||
from .provision import supervise as _supervise
|
||||
|
||||
|
||||
class SmolmachinesBottleBackend(
|
||||
@@ -56,26 +52,29 @@ class SmolmachinesBottleBackend(
|
||||
yield bottle
|
||||
|
||||
def provision_ca(
|
||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_ca.provision_ca(plan, bottle)
|
||||
_ca.provision_ca(plan, target)
|
||||
|
||||
def provision_workspace(
|
||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||
def provision_prompt(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> str | None:
|
||||
return _prompt.provision_prompt(plan, target)
|
||||
|
||||
def provision_skills(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_workspace.provision_workspace(plan, bottle)
|
||||
_skills.provision_skills(plan, target)
|
||||
|
||||
def provision_git(
|
||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_git.provision_git(plan, bottle)
|
||||
_git.provision_git(plan, target)
|
||||
|
||||
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
||||
"""The smolmachines guest reaches the supervise sidecar via a
|
||||
host-published random port the launch step pinned earlier
|
||||
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
|
||||
on the plan is "" when the bottle has no sidecar."""
|
||||
return plan.agent_supervise_url
|
||||
def provision_supervise(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_supervise.provision_supervise(plan, target)
|
||||
|
||||
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
@@ -45,11 +45,19 @@ _HOME_FOR = {
|
||||
}
|
||||
|
||||
|
||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||
def _env_flags_for(user: str) -> list[str]:
|
||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||
out = [f"HOME={home}", f"USER={user}"]
|
||||
return ["-e", f"HOME={home}", "-e", f"USER={user}"]
|
||||
|
||||
|
||||
def _guest_env_flags(env: Mapping[str, str]) -> list[str]:
|
||||
"""Render `{K: V}` into a flat `-e K=V` argv slice for
|
||||
`smolvm machine exec`. `smolvm machine create -e` set env
|
||||
on PID 1 but it doesn't propagate to fresh exec process
|
||||
trees, so we have to re-pass them every call."""
|
||||
out: list[str] = []
|
||||
for k, v in env.items():
|
||||
out.append(f"{k}={v}")
|
||||
out += ["-e", f"{k}={v}"]
|
||||
return out
|
||||
|
||||
|
||||
@@ -90,8 +98,9 @@ class SmolmachinesBottle(Bottle):
|
||||
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
||||
if tty:
|
||||
flags += ["-i", "-t"]
|
||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
|
||||
self.agent_command]
|
||||
flags += _env_flags_for("node")
|
||||
flags += _guest_env_flags(self._guest_env)
|
||||
agent_tail = [self.agent_command]
|
||||
provider_prompt_args = prompt_args(
|
||||
self._agent_prompt_mode, self._prompt_path, argv=argv,
|
||||
)
|
||||
@@ -139,16 +148,16 @@ class SmolmachinesBottle(Bottle):
|
||||
on both backends. Pass `user="root"` for tests that need
|
||||
root.
|
||||
|
||||
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
||||
without invoking a login shell, then sets HOME / USER and the
|
||||
bottle env in the child process."""
|
||||
argv = [
|
||||
"--", "runuser", "-u", user, "--",
|
||||
"env", *_env_assignments_for(user, self._guest_env),
|
||||
"/bin/sh", "-c", script,
|
||||
]
|
||||
# Call smolvm directly because this path needs the host-side
|
||||
# subprocess capture shape used by the Docker backend.
|
||||
`runuser -u <user> -- /bin/sh -c <script>` switches UID
|
||||
without invoking a login shell; HOME / USER are set via
|
||||
`smolvm -e` (see `_env_flags_for`)."""
|
||||
argv = (
|
||||
_env_flags_for(user)
|
||||
+ _guest_env_flags(self._guest_env)
|
||||
+ ["--", "runuser", "-u", user, "--", "/bin/sh", "-c", script]
|
||||
)
|
||||
# _smolvm.machine_exec expects argv (the bit after `--`);
|
||||
# the -e flags go before, so call smolvm directly.
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
||||
capture_output=True, text=True, check=False,
|
||||
|
||||
@@ -8,20 +8,25 @@ in chunk 4."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from ...egress import EgressPlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...log import info
|
||||
from ...pipelock import PipelockProxyPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottlePlan
|
||||
from ..print_util import print_multi, visible_agent_env_names
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SmolmachinesBottlePlan(BottlePlan):
|
||||
"""Resolved fields the launch step needs to bring up the bottle.
|
||||
|
||||
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
|
||||
`supervise_plan`, and `agent_provision` from BottlePlan."""
|
||||
Inherits `spec` and `stage_dir` from BottlePlan."""
|
||||
|
||||
slug: str
|
||||
# Per-bottle docker subnet for the sidecar bundle container.
|
||||
@@ -63,7 +68,7 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
# empty when the agent has no prompt — claude-code reads it
|
||||
# via --append-system-prompt-file only when non-empty.
|
||||
prompt_file: Path
|
||||
# Inner Plans for the sidecar bundle daemons. The same shape the
|
||||
# Inner Plans for the four bundle daemons. The same shape the
|
||||
# docker backend uses — same `.prepare()` calls produced
|
||||
# them — but our launch step doesn't populate the
|
||||
# docker-specific network fields (internal_network,
|
||||
@@ -72,6 +77,11 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||
# at their dataclass defaults.
|
||||
proxy_plan: PipelockProxyPlan
|
||||
git_gate_plan: GitGatePlan
|
||||
egress_plan: EgressPlan
|
||||
# None when bottle.supervise is False, matching the docker
|
||||
# backend's convention.
|
||||
supervise_plan: SupervisePlan | None
|
||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||
# networking; docker container IPs live in the daemon's VM),
|
||||
@@ -83,19 +93,40 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_host: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
agent_command: str = "claude"
|
||||
agent_prompt_mode: PromptMode = "append_file"
|
||||
agent_provider_template: str = "claude"
|
||||
agent_dockerfile_path: str = ""
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Compact y/N preflight. Same shape as the Docker
|
||||
backend's so operators see one format across backends."""
|
||||
del remote_control # not surfaced in the compact summary
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(bottle.env.keys()),
|
||||
agent_provider_template=self.agent_provider_template,
|
||||
)
|
||||
upstreams = [
|
||||
f"{g.Name} → {g.Upstream}" for g in bottle.git
|
||||
]
|
||||
# Use the resolved egress_plan (lowercase `host` on the
|
||||
# plan-level EgressRoute) rather than `bottle.egress.routes`,
|
||||
# which is the manifest's capitalized-attr form.
|
||||
routes = [r.host for r in self.egress_plan.routes]
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def agent_dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provider_template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
if upstreams:
|
||||
print_multi(" git gate ", upstreams)
|
||||
if routes:
|
||||
print_multi(" egress ", routes)
|
||||
print(file=sys.stderr)
|
||||
|
||||
@@ -21,14 +21,12 @@ from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import time
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import (
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
egress_resolve_token_values,
|
||||
)
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
|
||||
from ...pipelock import (
|
||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||
@@ -47,15 +45,13 @@ from ..docker.git_gate import (
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_PORT as _GIT_GATE_PORT,
|
||||
)
|
||||
from ..docker.pipelock import (
|
||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
|
||||
pipelock_tls_init,
|
||||
)
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import warn
|
||||
from ..docker.bottle_state import git_gate_state_dir
|
||||
from . import loopback_alias as _loopback
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
@@ -81,7 +77,6 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
|
||||
# them up post-start. Pipelock's port is an env-overridable string
|
||||
# in docker.pipelock; coerce to int here.
|
||||
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
|
||||
_GIT_HTTP_PORT = 9420
|
||||
_SUPERVISE_PORT = SUPERVISE_PORT
|
||||
|
||||
|
||||
@@ -96,230 +91,210 @@ def launch(
|
||||
via the ExitStack."""
|
||||
stack = ExitStack()
|
||||
try:
|
||||
loopback_ip, network = _allocate_resources(plan, stack)
|
||||
plan = _mint_certs(plan)
|
||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||
plan = _discover_urls(plan, loopback_ip)
|
||||
# 1. Reserve a loopback alias for this bottle. macOS only
|
||||
# routes 127.0.0.1 by default; the per-bottle alias is
|
||||
# what bundles the docker port-publishes and TSI allowlist
|
||||
# against, so this bottle can't reach other bottles' (or
|
||||
# other host services') ports on the loopback. Lazy
|
||||
# sudo-driven on first use per boot. No-op on Linux.
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs
|
||||
# here, not in prepare, so the docker-build output doesn't
|
||||
# garble the dashboard's preflight modal.
|
||||
# 2. Per-bottle docker bridge.
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
|
||||
# 2. Mint per-bottle CAs and update the inner Plans with
|
||||
# their launch-time paths. pipelock always runs in the
|
||||
# bundle; egress's CA is only minted when the bottle
|
||||
# declares routes (otherwise egress runs idle without
|
||||
# MITM and the CA files would be unused).
|
||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
||||
proxy_plan = dataclasses.replace(
|
||||
plan.proxy_plan,
|
||||
ca_cert_host_path=ca_cert_host,
|
||||
ca_key_host_path=ca_key_host,
|
||||
)
|
||||
egress_plan = plan.egress_plan
|
||||
if egress_plan.routes:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
plan.egress_plan.routes_path.parent,
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
pipelock_ca_host_path=ca_cert_host,
|
||||
# On smolmachines, egress's upstream is pipelock
|
||||
# on the bundle's localhost — they're in the same
|
||||
# container's network namespace.
|
||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
)
|
||||
plan = dataclasses.replace(
|
||||
plan, proxy_plan=proxy_plan, egress_plan=egress_plan,
|
||||
)
|
||||
|
||||
# 3. Build the BundleLaunchSpec from the (now-resolved)
|
||||
# inner Plans: daemon subset, env, bind-mounts, and the
|
||||
# loopback alias to bind published ports against. The
|
||||
# spec's ports_to_publish list expands depending on which
|
||||
# daemons the agent needs to reach from the smolvm guest.
|
||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||
token_env = _resolve_token_env(plan, os.environ)
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
|
||||
# 4. Discover the host-side ports docker assigned for the
|
||||
# bundle's published container ports, and bind the
|
||||
# agent's URLs to `<loopback_ip>:<host port>`. Docker
|
||||
# container IPs (192.168.x.x in the daemon's bridge)
|
||||
# aren't reachable from the smolvm guest on macOS — TSI
|
||||
# uses macOS networking, and macOS sees the daemon's
|
||||
# bridge via the published-port loopback forward only.
|
||||
#
|
||||
# Proxy hop order matches the docker backend: when the
|
||||
# bottle declares egress routes, the agent's first hop is
|
||||
# egress (for token injection), then pipelock. Without
|
||||
# routes, the agent dials pipelock directly. Whichever
|
||||
# one is "agent-facing" is the daemon whose port we
|
||||
# publish on host loopback; the other stays bundle-
|
||||
# internal as the upstream proxy.
|
||||
if plan.egress_plan.routes:
|
||||
agent_facing_port = _EGRESS_PORT
|
||||
else:
|
||||
agent_facing_port = _PIPELOCK_PORT
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
||||
)
|
||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_GATE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
# Stamp the URLs onto the plan + guest_env. provision_git
|
||||
# and provision_supervise read the plan fields; the agent
|
||||
# reads guest_env on every exec_agent.
|
||||
#
|
||||
# NO_PROXY has to include the per-bottle loopback alias —
|
||||
# otherwise claude's HTTPS_PROXY catches direct calls to
|
||||
# the supervise URL (`http://<alias>:<port>/`) and proxies
|
||||
# them through egress, which has no route for the alias
|
||||
# and rejects with "Failed to connect". The git-gate URL
|
||||
# uses git://, not affected by HTTP_PROXY, so the alias
|
||||
# only has to be in NO_PROXY for the MCP / supervise
|
||||
# path. Append rather than overwrite so prepare.py's
|
||||
# `localhost,127.0.0.1` baseline stays in place.
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"git://{agent_git_gate_host}"
|
||||
if agent_supervise_url:
|
||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
guest_env=guest_env,
|
||||
agent_proxy_url=agent_proxy_url,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
)
|
||||
|
||||
# 5. Build the agent image and pack it into a
|
||||
# `.smolmachine` artifact (or hit the per-Dockerfile-digest
|
||||
# cache). Runs here, not in prepare, so the docker-build
|
||||
# output doesn't garble the dashboard's preflight modal:
|
||||
# both the curses-endwin path and the tmux pane-routing
|
||||
# path redirect stderr around `launch` already.
|
||||
agent_from_path = _ensure_smolmachine(
|
||||
plan.agent_image_ref,
|
||||
dockerfile=plan.agent_dockerfile_path,
|
||||
)
|
||||
|
||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||
_init_vm(plan)
|
||||
|
||||
bottle = SmolmachinesBottle(
|
||||
# smolvm VM. --from carries the pre-packed .smolmachine
|
||||
# artifact; --allow-cidr + -e carry the per-bottle TSI
|
||||
# allowlist + env. The allowlist is the per-bottle
|
||||
# loopback alias — narrowing it to one /32 keeps the
|
||||
# agent from reaching other host loopback services or
|
||||
# other bottles' published ports. Smolfile isn't usable
|
||||
# here — smolvm 0.8.0 makes `--from` and `--smolfile`
|
||||
# mutually exclusive.
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
prompt_path=None,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[f"{loopback_ip}/32"],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently
|
||||
# dropped when combined with `--from`. Patch the persisted
|
||||
# state DB to set the allowlist before start so the booted
|
||||
# VM's TSI actually enforces. See loopback_alias's module
|
||||
# docstring for the investigation that led here.
|
||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
# 6. Repair filesystem ownership + perms that smolvm's
|
||||
# pack process remapped to the host invoker's uid (501
|
||||
# on macOS) rather than preserving the image's expected
|
||||
# ownership.
|
||||
#
|
||||
# - /home/node → node:node so the node user can write
|
||||
# its own dotfiles (claude appendFileSync on
|
||||
# ~/.claude.json otherwise bails with ENOENT/EPERM
|
||||
# and the TUI hangs without surfacing the error).
|
||||
# - /tmp + /var/tmp → root:root mode 1777 so non-root
|
||||
# processes can create their per-uid scratch dirs
|
||||
# (claude-code creates /tmp/claude-<uid>/ as soon as
|
||||
# it spawns a Bash tool call).
|
||||
#
|
||||
# All folded into one sh -c so we only pay one
|
||||
# machine_exec round trip — back-to-back exec calls
|
||||
# right after machine_start hit a SIGKILL race in
|
||||
# libkrun's exec channel (see provision_ca for the
|
||||
# other half of this same workaround).
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
])
|
||||
|
||||
# Wait briefly for the VM to settle. Back-to-back smolvm
|
||||
# machine_exec calls immediately after machine_start
|
||||
# occasionally SIGKILL the in-VM child at ~100ms (looks
|
||||
# like a VM warm-up race in libkrun's exec channel).
|
||||
# 1.5s is empirically enough to dodge it; provisioning
|
||||
# already takes seconds so the wait is amortized.
|
||||
time.sleep(1.5)
|
||||
|
||||
# 7. Provision (CA / prompt / skills / git / supervise).
|
||||
prompt_path = provision(plan, plan.machine_name)
|
||||
|
||||
yield SmolmachinesBottle(
|
||||
plan.machine_name,
|
||||
prompt_path=prompt_path,
|
||||
guest_env=plan.guest_env,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
)
|
||||
bottle._prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
_teardown_smolmachines(stack, plan)
|
||||
|
||||
|
||||
def _teardown_smolmachines(
|
||||
stack: ExitStack,
|
||||
plan: SmolmachinesBottlePlan,
|
||||
) -> None:
|
||||
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
|
||||
|
||||
ExitStack errors are caught and logged (non-fatal) so that key
|
||||
revocation always runs. Revocation errors propagate — a stranded
|
||||
deploy key is a security concern the operator must address."""
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc:
|
||||
teardown_exc = exc
|
||||
warn(f"smolmachines teardown failed: {exc!r}")
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
|
||||
def _allocate_resources(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
stack: ExitStack,
|
||||
) -> tuple[str, str]:
|
||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||
|
||||
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
||||
scopes TSI's allowlist to this bottle's published ports so the
|
||||
agent can't reach other bottles' or host services' ports on
|
||||
loopback. No-op on Linux."""
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
return loopback_ip, network
|
||||
|
||||
|
||||
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||
"""Mint per-bottle CAs and return the plan with CA paths filled.
|
||||
|
||||
Pipelock always runs in the bundle. Egress's CA is only minted
|
||||
when the bottle declares routes — otherwise egress runs idle
|
||||
without MITM and the CA files would be unused."""
|
||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
||||
proxy_plan = dataclasses.replace(
|
||||
plan.proxy_plan,
|
||||
ca_cert_host_path=ca_cert_host,
|
||||
ca_key_host_path=ca_key_host,
|
||||
)
|
||||
egress_plan = plan.egress_plan
|
||||
if egress_plan.routes:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
plan.egress_plan.routes_path.parent,
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
pipelock_ca_host_path=ca_cert_host,
|
||||
# On smolmachines, egress's upstream is pipelock on the
|
||||
# bundle's localhost — they're in the same container's
|
||||
# network namespace.
|
||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
)
|
||||
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _start_bundle(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
network: str,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||
sidecar bundle container, and register teardown."""
|
||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
return plan
|
||||
|
||||
|
||||
def _discover_urls(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
loopback_ip: str,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Discover host-side ports for published container ports and
|
||||
return the plan with URLs + guest_env stamped in.
|
||||
|
||||
Docker container IPs (192.168.x.x in the daemon's bridge)
|
||||
aren't reachable from the smolvm guest on macOS — TSI uses
|
||||
macOS networking, and macOS sees the daemon's bridge via the
|
||||
published-port loopback forward only.
|
||||
|
||||
Proxy hop order: when the bottle declares egress routes, the
|
||||
agent's first hop is egress (for token injection), then
|
||||
pipelock. Without routes, the agent dials pipelock directly.
|
||||
NO_PROXY includes the per-bottle loopback alias so the
|
||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
||||
if plan.egress_plan.routes:
|
||||
agent_facing_port = _EGRESS_PORT
|
||||
else:
|
||||
agent_facing_port = _PIPELOCK_PORT
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
||||
)
|
||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||
if agent_supervise_url:
|
||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
guest_env=guest_env,
|
||||
agent_proxy_url=agent_proxy_url,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
)
|
||||
|
||||
|
||||
def _launch_vm(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
agent_from_path: Path,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
"""Create, patch, and start the smolvm VM; register teardown.
|
||||
|
||||
--allow-cidr is the per-bottle loopback alias so the guest can
|
||||
only reach this bottle's bundle ports. force_allowlist patches
|
||||
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
||||
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||
and --smolfile mutually exclusive."""
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[f"{loopback_ip}/32"],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
||||
# when combined with `--from`. Patch the persisted state DB
|
||||
# before start so the booted VM's TSI actually enforces.
|
||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
|
||||
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||
|
||||
Ownership repair: smolvm's pack process remaps files to the host
|
||||
invoker's uid (501 on macOS). /home/node must be node:node so
|
||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||
All folded into one sh -c to avoid back-to-back exec calls
|
||||
immediately after machine_start (libkrun exec-channel race).
|
||||
|
||||
wait_exec_ready polls until the exec channel is ready for the
|
||||
subsequent provision calls, replacing the empirical sleep."""
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
])
|
||||
_smolvm.wait_exec_ready(plan.machine_name)
|
||||
|
||||
|
||||
def _bundle_launch_spec(
|
||||
@@ -330,10 +305,10 @@ def _bundle_launch_spec(
|
||||
Daemons in the CSV:
|
||||
- egress + pipelock are always present (pipelock is the
|
||||
agent's first hop; egress is its upstream).
|
||||
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
||||
- git-gate is conditional on plan.git_gate_plan.upstreams.
|
||||
- supervise is conditional on plan.supervise_plan.
|
||||
|
||||
Env + volumes are the union of the sidecar daemons' needs, with
|
||||
Env + volumes are the union of the four daemons' needs, with
|
||||
daemon-private values only (HTTPS_PROXY is scoped to the
|
||||
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
||||
bind-address PR)."""
|
||||
@@ -345,9 +320,10 @@ def _bundle_launch_spec(
|
||||
# is "agent-facing" gets its port published on the host
|
||||
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
||||
# other stays bundle-internal. The bundle is NOT reachable by
|
||||
# bridge IP from the smolvm guest on macOS — TSI uses macOS
|
||||
# networking, and macOS sees the daemon's bridge via the
|
||||
# published-port loopback forward only.
|
||||
# bridge IP from the smolvm guest, so the
|
||||
# PRD-0023-chunk-3 EGRESS_LISTEN_HOST=127.0.0.1 mitigation
|
||||
# isn't needed: the agent can only dial whatever daemon's
|
||||
# host port we publish, period.
|
||||
|
||||
# --- pipelock ---------------------------------------------
|
||||
pp = plan.proxy_plan
|
||||
@@ -374,9 +350,10 @@ def _bundle_launch_spec(
|
||||
env.append(token_env)
|
||||
|
||||
# --- git-gate ---------------------------------------------
|
||||
extra_hosts: list[str] = []
|
||||
gp = plan.git_gate_plan
|
||||
if gp.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
daemons.append("git-gate")
|
||||
volumes += [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
|
||||
@@ -418,7 +395,7 @@ def _bundle_launch_spec(
|
||||
else:
|
||||
ports_to_publish = [_PIPELOCK_PORT]
|
||||
if gp.upstreams:
|
||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||
ports_to_publish.append(_GIT_GATE_PORT)
|
||||
if sp is not None:
|
||||
ports_to_publish.append(_SUPERVISE_PORT)
|
||||
|
||||
@@ -437,13 +414,15 @@ def _bundle_launch_spec(
|
||||
|
||||
|
||||
def _resolve_token_env(
|
||||
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
|
||||
plan: SmolmachinesBottlePlan, host_env: object
|
||||
) -> dict[str, str]:
|
||||
"""Resolve the egress token env-var values from the host's
|
||||
environ so they reach the bundle's process env via docker's
|
||||
`-e NAME` inheritance. Empty when no routes declare auth."""
|
||||
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
|
||||
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||
ep = plan.egress_plan
|
||||
if not ep.routes:
|
||||
return {}
|
||||
return egress_resolve_token_values(ep.token_env_map, dict(host_env))
|
||||
|
||||
|
||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
|
||||
@@ -45,7 +45,6 @@ alias gets handed to a new bottle."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
@@ -84,14 +83,6 @@ _POOL_START = 16
|
||||
_POOL_END = 31 # inclusive
|
||||
|
||||
|
||||
# File lock that serialises concurrent allocate() calls so two
|
||||
# simultaneous launches can't read the same docker state and claim
|
||||
# the same alias. Narrowed to the allocate() call itself; docker run
|
||||
# runs after the lock is released. Once the container is running it
|
||||
# appears in docker state and future allocate() calls will see it.
|
||||
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
||||
|
||||
|
||||
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
||||
def _pool_addresses() -> list[str]:
|
||||
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
||||
@@ -188,20 +179,9 @@ def allocate(slug: str) -> str:
|
||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
||||
This still returns a deterministic address so launch.py's
|
||||
callers don't have to branch on platform.
|
||||
|
||||
An exclusive file lock serialises concurrent calls so two
|
||||
simultaneous launches don't read the same docker state and
|
||||
claim the same alias."""
|
||||
callers don't have to branch on platform."""
|
||||
if not _is_macos():
|
||||
return "127.0.0.1"
|
||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||
return _allocate_locked()
|
||||
|
||||
|
||||
def _allocate_locked() -> str:
|
||||
in_use = _aliases_in_use()
|
||||
for ip in _pool_addresses():
|
||||
if ip not in in_use:
|
||||
|
||||
@@ -12,10 +12,9 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import agent_provision_plan, runtime_for
|
||||
from ...agent_provider import runtime_for
|
||||
from ...backend import BottleSpec
|
||||
from ...backend.docker.bottle_state import (
|
||||
BottleMetadata,
|
||||
@@ -28,11 +27,9 @@ from ...backend.docker.bottle_state import (
|
||||
write_metadata,
|
||||
)
|
||||
from ...egress import Egress
|
||||
from ...env import resolve_env
|
||||
from ...git_gate import GitGate
|
||||
from ...pipelock import PipelockProxy
|
||||
from ...supervise import Supervise
|
||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
@@ -61,8 +58,6 @@ def resolve_plan(
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
guest_home = "/home/node"
|
||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||
|
||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||
|
||||
@@ -74,34 +69,72 @@ def resolve_plan(
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
# No compose project for smolmachines bottles; chunk 4
|
||||
# will give dashboard discovery a backend-specific path.
|
||||
compose_project="",
|
||||
backend="smolmachines",
|
||||
))
|
||||
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||
# values still reach the guest as -e K=V smolvm flags because
|
||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||
# the known argv-exposure gap documented in PRD 0038.
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||
# in launch.py after bundle bringup.
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
# Agent's env: the prepare-time view doesn't yet know the
|
||||
# host loopback ports the bundle's daemons get published on
|
||||
# (those come from docker AFTER `docker run` returns), so
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are
|
||||
# populated in launch.py and stamped onto guest_env there.
|
||||
# What we set here is the part that doesn't depend on
|
||||
# bundle bringup — bottle.env literals, the empty-NO_PROXY
|
||||
# safe default, and the TLS trust env trio
|
||||
# (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE)
|
||||
# pointing at Debian's update-ca-certificates output bundle.
|
||||
guest_env: dict[str, str] = {
|
||||
**resolved.literals,
|
||||
**resolved.forwarded,
|
||||
**bottle.env,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
# Inner Plans for the four bundle daemons. The ABCs are
|
||||
# platform-neutral — `.prepare()` writes config files + returns
|
||||
# a Plan dataclass with no backend-specific assumptions. State
|
||||
# dirs are still keyed by slug under the docker backend's
|
||||
# bottle_state layout (shared on-host convention; not a docker
|
||||
# dependency).
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = PipelockProxy().prepare(bottle, slug, pipelock_dir)
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = Egress().prepare(bottle, slug, egress_dir)
|
||||
|
||||
# Claude-code refuses to start without *something* it
|
||||
# recognises as a credential. When the bottle has an egress
|
||||
# route carrying the `claude_code_oauth` role marker, egress
|
||||
# strips + re-injects the real Authorization header on the
|
||||
# outbound leg using a token held in egress's own environ — so
|
||||
# the agent gets a non-secret placeholder here (matches the
|
||||
# docker backend's forwarded_env logic in
|
||||
# bot_bottle/backend/docker/prepare.py).
|
||||
has_provider_auth = any(
|
||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
||||
)
|
||||
if has_provider_auth:
|
||||
guest_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
||||
if provider.template == "claude" and has_provider_auth:
|
||||
guest_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
||||
guest_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
# Prompt file is always written (mode 0o600) so the in-VM
|
||||
# path always exists. Content is the agent's `prompt`
|
||||
# field (markdown body) — empty for agents with no prompt.
|
||||
@@ -129,64 +162,26 @@ def resolve_plan(
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||
agent_provision = agent_provision_plan(
|
||||
template=provider.template,
|
||||
dockerfile=agent_dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
guest_home=guest_home,
|
||||
guest_env=guest_env,
|
||||
forward_host_credentials=provider.forward_host_credentials,
|
||||
auth_token=provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace_plan.workdir,
|
||||
)
|
||||
merged_guest_env = dict(agent_provision.guest_env)
|
||||
for key, val in agent_provision.env_vars.items():
|
||||
merged_guest_env.setdefault(key, val)
|
||||
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
|
||||
|
||||
# Inner Plans for the four bundle daemons. The ABCs are
|
||||
# platform-neutral — `.prepare()` writes config files + returns
|
||||
# a Plan dataclass with no backend-specific assumptions. State
|
||||
# dirs are still keyed by slug under the docker backend's
|
||||
# bottle_state layout (shared on-host convention; not a docker
|
||||
# dependency).
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = PipelockProxy().prepare(
|
||||
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
||||
)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = Egress().prepare(
|
||||
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||
)
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
guest_home=guest_home,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
machine_name=machine_name,
|
||||
agent_image_ref=agent_image_ref,
|
||||
guest_env=agent_provision.guest_env,
|
||||
guest_env=guest_env,
|
||||
prompt_file=prompt_file,
|
||||
proxy_plan=proxy_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision,
|
||||
workspace_plan=workspace_plan,
|
||||
agent_command=provider_runtime.command,
|
||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
||||
agent_provider_template=provider.template,
|
||||
agent_dockerfile_path=agent_dockerfile_path,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
"""Backend-infrastructure provisioners for the smolmachines backend.
|
||||
"""Provisioning helpers for the smolmachines backend (PRD 0023
|
||||
chunk 4).
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration) live on
|
||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
|
||||
left in this subpackage handle only the steps that are
|
||||
backend-specific:
|
||||
Each method maps onto one of `BottleBackend`'s `provision_*`
|
||||
overrides. They run after the VM is up + the bundle is reachable
|
||||
and copy host-side state (prompt, skills, .git, CA cert,
|
||||
supervise MCP config) into the guest via `smolvm machine cp` /
|
||||
`smolvm machine exec`.
|
||||
|
||||
- ca.py — install per-bottle CA bundle into the guest trust store
|
||||
- git.py — copy host cwd `.git` into the guest when --cwd is used
|
||||
- workspace.py — copy the operator workspace into the guest
|
||||
"""
|
||||
Chunk 4a ships `provision_prompt` and `provision_skills` — the
|
||||
two that don't depend on agent-image tooling (claude-code,
|
||||
update-ca-certificates) beyond `cp` and `mkdir`. provision_ca /
|
||||
provision_git / provision_supervise land once the agent-image
|
||||
gap is solved."""
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
trust store (PRD 0023 chunk 4d).
|
||||
|
||||
Mirrors `backend.docker.provision.ca`: select the right CA (egress
|
||||
when the bottle has routes, else pipelock), copy it to Debian's
|
||||
`/usr/local/share/ca-certificates/` path,
|
||||
when the bottle has routes, else pipelock), `smolvm machine cp` it
|
||||
to Debian's `/usr/local/share/ca-certificates/` path,
|
||||
`update-ca-certificates` to rebuild the trust bundle, and log the
|
||||
fingerprint once. The selected cert depends on the agent's
|
||||
HTTP_PROXY target — same logic as the docker backend, since the
|
||||
@@ -15,8 +15,6 @@ flag exists; the VM init is root), so we don't need the explicit
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from ....log import die
|
||||
from ...util import (
|
||||
AGENT_CA_BUNDLE,
|
||||
@@ -24,20 +22,17 @@ from ...util import (
|
||||
log_ca_fingerprint,
|
||||
select_ca_cert,
|
||||
)
|
||||
from ... import Bottle, ExecResult
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
_SIGKILL_EXIT = 128 + 9
|
||||
|
||||
|
||||
def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Copy the agent-facing CA cert into the guest, rebuild the
|
||||
trust bundle, emit a one-line fingerprint log. Called from
|
||||
`BottleBackend.provision` after the smolvm guest is up."""
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||
|
||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||
_smolvm.machine_cp(str(cert_host_path), f"{target}:{AGENT_CA_PATH}")
|
||||
# Mode 0644 — readable to non-root tools in the guest.
|
||||
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
|
||||
# which is what curl / Python ssl / OpenSSL-based tools read by
|
||||
@@ -45,21 +40,22 @@ def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
||||
# `requests` / libraries that don't load the system bundle.
|
||||
#
|
||||
r = _install_ca(bottle)
|
||||
if r.returncode == _SIGKILL_EXIT:
|
||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec
|
||||
# during early-VM provisioning. `update-ca-certificates`
|
||||
# is idempotent, so retry the same install once after a
|
||||
# short settle delay before treating it as fatal.
|
||||
time.sleep(1.0)
|
||||
r = _install_ca(bottle)
|
||||
|
||||
if r.returncode != 0:
|
||||
# chown + chmod + update-ca-certificates run in one
|
||||
# `sh -c` so we only pay one machine_exec round trip; the
|
||||
# `&&` chaining surfaces the first failure as the return
|
||||
# code.
|
||||
r = _smolvm.machine_exec(target, [
|
||||
"sh", "-c",
|
||||
f"chown root:root {AGENT_CA_PATH} && "
|
||||
f"chmod 644 {AGENT_CA_PATH} && "
|
||||
f"update-ca-certificates",
|
||||
])
|
||||
if r.returncode != 0 or "1 added" not in (r.stdout or ""):
|
||||
# update-ca-certificates not adding our cert is fatal —
|
||||
# claude-code's TLS handshake against the egress-MITM'd
|
||||
# api.anthropic.com would fail downstream. Bail early
|
||||
# with what we can see (output is captured so we can
|
||||
# surface it).
|
||||
# with what we can see (output is captured by smolvm so
|
||||
# we can surface it).
|
||||
die(
|
||||
f"update-ca-certificates didn't add the agent CA "
|
||||
f"(exit {r.returncode}): "
|
||||
@@ -70,23 +66,6 @@ def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
log_ca_fingerprint(cert_host_path, label)
|
||||
|
||||
|
||||
def _install_ca(bottle: Bottle) -> ExecResult:
|
||||
# chown + chmod + update-ca-certificates + bundle
|
||||
# verification run in one exec so we only pay one
|
||||
# round trip; the `&&` chaining surfaces the first failure
|
||||
# as the return code. The verify check is more stable than
|
||||
# requiring "1 added" in stdout: a retry after a
|
||||
# partially-completed first run may legitimately report "0
|
||||
# added" while the cert is already installed.
|
||||
return bottle.exec(
|
||||
f"chown root:root {AGENT_CA_PATH} && "
|
||||
f"chmod 644 {AGENT_CA_PATH} && "
|
||||
f"update-ca-certificates && "
|
||||
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
|
||||
user="root",
|
||||
)
|
||||
|
||||
|
||||
# Re-exported for the launch/provision_ca caller + tests. The path
|
||||
# constants live in the shared `backend.util` (Debian's
|
||||
# `update-ca-certificates` layout is the same in both backends).
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
Three concerns, all about git in the agent:
|
||||
|
||||
1. If --cwd was passed AND the host cwd has a .git, copy that
|
||||
.git into the planned guest workspace so the agent operates on
|
||||
.git into /home/node/workspace/.git so the agent operates on
|
||||
the user's repo.
|
||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||
~/.gitconfig with insteadOf rules so every git operation
|
||||
@@ -18,7 +18,7 @@ Three concerns, all about git in the agent:
|
||||
Differs from `backend.docker.provision.git` in one address detail:
|
||||
the TSI-allowlisted guest can only reach the bundle's pinned IP
|
||||
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
|
||||
are `http://<bundle_ip>:<port>/<name>.git` rather than the
|
||||
are `git://<bundle_ip>:<port>/<name>.git` rather than the
|
||||
docker backend's `git://git-gate/<name>.git`. The render itself
|
||||
is the shared `git_gate_render_gitconfig` on the platform-neutral
|
||||
git_gate module."""
|
||||
@@ -26,66 +26,71 @@ git_gate module."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from ....git_gate import git_gate_render_gitconfig
|
||||
from ....log import info
|
||||
from ... import Bottle
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
def provision_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
# `node` is the agent user from the repo Dockerfile. Override via
|
||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||
# BOT_BOTTLE_CONTAINER_HOME knob — same purpose, different
|
||||
# transport.
|
||||
_DEFAULT_GUEST_HOME = "/home/node"
|
||||
|
||||
|
||||
def _guest_home() -> str:
|
||||
return os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||
|
||||
|
||||
def provision_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Set up git inside the guest. Runs all three subcases; each
|
||||
no-ops when its condition isn't met."""
|
||||
_provision_cwd_git(plan, bottle)
|
||||
_provision_git_gate_config(plan, bottle)
|
||||
_provision_git_user(plan, bottle)
|
||||
_provision_cwd_git(plan, target)
|
||||
_provision_git_gate_config(plan, target)
|
||||
_provision_git_user(plan, target)
|
||||
|
||||
|
||||
def _provision_cwd_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||
it into <guest_home>/workspace/.git and fix ownership. No-op
|
||||
otherwise."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||
return
|
||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||
host_git = str(workspace.host_path / ".git")
|
||||
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
||||
# mkdir -p the workspace dir so cp_in lands the .git
|
||||
guest_workspace_git = f"{_guest_home()}/workspace/.git"
|
||||
info(f"copying {plan.spec.user_cwd}/.git -> {target}:{guest_workspace_git}")
|
||||
# mkdir -p the workspace dir so `machine cp` lands the .git
|
||||
# directly there even on first-time bottles.
|
||||
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
|
||||
bottle.cp_in(host_git, guest_workspace_git)
|
||||
# cp_in lands files as root; the agent runs as node so
|
||||
_smolvm.machine_exec(target, ["mkdir", "-p", f"{_guest_home()}/workspace"])
|
||||
_smolvm.machine_cp(
|
||||
f"{plan.spec.user_cwd}/.git", f"{target}:{guest_workspace_git}",
|
||||
)
|
||||
# `machine cp` lands files as root; the agent runs as node so
|
||||
# the workspace tree must be chowned over.
|
||||
bottle.exec(
|
||||
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
|
||||
user="root",
|
||||
_smolvm.machine_exec(
|
||||
target, ["chown", "-R", "node:node", guest_workspace_git],
|
||||
)
|
||||
|
||||
|
||||
def _provision_git_gate_config(
|
||||
plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||
) -> None:
|
||||
def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
|
||||
rules. No-op when the bottle has no `git` entries."""
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not manifest_bottle.git:
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not bottle.git:
|
||||
return
|
||||
|
||||
# `<loopback alias>:<host port>` form: the bundle's git-gate
|
||||
# HTTP port is published on host loopback at launch time so
|
||||
# the smolvm guest (which can only reach macOS networking via
|
||||
# `127.0.0.1:<host port>` form: the bundle's git-gate port
|
||||
# is published on host loopback at launch time so the
|
||||
# smolvm guest (which can only reach macOS networking via
|
||||
# TSI, not the docker bridge IP) can dial it. launch.py
|
||||
# populates `plan.agent_git_gate_host` after bundle bringup.
|
||||
content = git_gate_render_gitconfig(
|
||||
manifest_bottle.git, plan.agent_git_gate_host, scheme="http",
|
||||
)
|
||||
content = git_gate_render_gitconfig(bottle.git, plan.agent_git_gate_host)
|
||||
|
||||
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
|
||||
# Stage the file under the plan's stage_dir so cp_in
|
||||
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
||||
# Stage the file under the plan's stage_dir so `machine cp`
|
||||
# has a stable host path. The plan's stage_dir is cleaned up
|
||||
# by start.py's session-end teardown.
|
||||
with tempfile.NamedTemporaryFile(
|
||||
@@ -96,38 +101,41 @@ def _provision_git_gate_config(
|
||||
config_file = Path(f.name)
|
||||
os.chmod(config_file, 0o600)
|
||||
|
||||
info(f"writing {guest_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
|
||||
bottle.cp_in(str(config_file), guest_gitconfig)
|
||||
bottle.exec(
|
||||
f"chown node:node {shlex.quote(guest_gitconfig)} && "
|
||||
f"chmod 644 {shlex.quote(guest_gitconfig)}",
|
||||
user="root",
|
||||
)
|
||||
info(f"writing {guest_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||
_smolvm.machine_cp(str(config_file), f"{target}:{guest_gitconfig}")
|
||||
_smolvm.machine_exec(target, ["chown", "node:node", guest_gitconfig])
|
||||
_smolvm.machine_exec(target, ["chmod", "644", guest_gitconfig])
|
||||
|
||||
|
||||
def _provision_git_user(
|
||||
plan: SmolmachinesBottlePlan, bottle: Bottle,
|
||||
plan: SmolmachinesBottlePlan, target: str,
|
||||
) -> None:
|
||||
"""Apply `git config --global user.{name,email}` inside the
|
||||
guest as the node user so --global lands in the same
|
||||
`/home/node/.gitconfig` that `_provision_git_gate_config`
|
||||
writes to. No-op when the bottle didn't declare `git.user`.
|
||||
|
||||
SmolmachinesBottle.exec(user="node") automatically sets
|
||||
HOME=/home/node so --global writes to /home/node/.gitconfig."""
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = manifest_bottle.git_user
|
||||
Runs via `runuser -u node --`; HOME is forced via smolvm's
|
||||
`-e` flag because runuser (without -l) inherits root's
|
||||
HOME=/root, which would put --global in the wrong file."""
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = bottle.git_user
|
||||
if gu.is_empty():
|
||||
return
|
||||
env = {"HOME": _guest_home(), "USER": "node"}
|
||||
if gu.name:
|
||||
info(f"git config --global user.name = {gu.name!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.name {shlex.quote(gu.name)}",
|
||||
user="node",
|
||||
_smolvm.machine_exec(
|
||||
target,
|
||||
["runuser", "-u", "node", "--",
|
||||
"git", "config", "--global", "user.name", gu.name],
|
||||
env=env,
|
||||
)
|
||||
if gu.email:
|
||||
info(f"git config --global user.email = {gu.email!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.email {shlex.quote(gu.email)}",
|
||||
user="node",
|
||||
_smolvm.machine_exec(
|
||||
target,
|
||||
["runuser", "-u", "node", "--",
|
||||
"git", "config", "--global", "user.email", gu.email],
|
||||
env=env,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Copy the agent prompt into a running smolmachines bottle.
|
||||
|
||||
The prompt file is always copied (so the in-guest path always
|
||||
exists) but `--append-system-prompt-file` only fires when the
|
||||
agent actually has a prompt — the return value signals which
|
||||
case, mirroring the docker backend's contract.
|
||||
|
||||
`smolvm machine cp` lands files as root inside the VM; the claude
|
||||
process runs as `node`, so we chown + chmod the prompt after the
|
||||
copy. Same flow as the docker backend's provision_prompt."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
# `node` is the agent user from the repo Dockerfile.
|
||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||
# BOT_BOTTLE_CONTAINER_HOME knob.
|
||||
_DEFAULT_GUEST_HOME = "/home/node"
|
||||
|
||||
|
||||
def provision_prompt(plan: SmolmachinesBottlePlan, target: str) -> str | None:
|
||||
"""Copy the prompt file into the running smolvm guest, fix
|
||||
ownership/mode. Returns the in-guest path if the agent has a
|
||||
non-empty prompt (drives --append-system-prompt-file), else
|
||||
None. The file is copied either way so the path always
|
||||
exists — mirrors the docker backend's behavior."""
|
||||
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||
in_guest_prompt_path = f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
_smolvm.machine_cp(str(plan.prompt_file), f"{target}:{in_guest_prompt_path}")
|
||||
# machine cp lands as root, source's 0o600 mode is preserved —
|
||||
# node can't read its own prompt without these two.
|
||||
_smolvm.machine_exec(target, ["chown", "node:node", in_guest_prompt_path])
|
||||
_smolvm.machine_exec(target, ["chmod", "600", in_guest_prompt_path])
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return in_guest_prompt_path if agent.prompt else None
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Copy host-side skill directories into a running smolmachines
|
||||
bottle.
|
||||
|
||||
Skills are validated on the host before launch by
|
||||
`BottleBackend._validate_skills`; this module assumes that
|
||||
validation has already run. A skill that disappears between
|
||||
validation and copy still dies loudly rather than silently
|
||||
producing a partial guest."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from ....log import die, info
|
||||
from ...util import host_skill_dir
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
# In-guest path mirrors the docker backend's claude-skills
|
||||
# convention (~/.claude/skills/<name>/) under the node user's
|
||||
# home — same path as the real bot-bottle image's
|
||||
# /home/node/.claude/skills (pre-created in the Dockerfile).
|
||||
_DEFAULT_SKILLS_DIR = "/home/node/.claude/skills"
|
||||
|
||||
|
||||
def provision_skills(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Copy each of the agent's named skills from the host's
|
||||
~/.claude/skills/<name>/ into the guest's equivalent path.
|
||||
For each skill: `mkdir -p` the destination, `smolvm machine cp`
|
||||
the host source dir over, then chown the result to node:node so
|
||||
the agent can read it. No-op when the agent has no skills.
|
||||
|
||||
smolvm machine cp on a directory copies recursively (same
|
||||
semantics as `cp -r`); unlike docker cp's trailing-slash
|
||||
convention, smolvm doesn't need the `/.` suffix dance.
|
||||
|
||||
machine cp lands files as root inside the VM, so we chown each
|
||||
skill tree over to node:node after the copy — same pattern as
|
||||
the docker backend's provision_prompt."""
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
|
||||
skills_dir = os.environ.get(
|
||||
"BOT_BOTTLE_GUEST_SKILLS_DIR", _DEFAULT_SKILLS_DIR,
|
||||
)
|
||||
|
||||
_smolvm.machine_exec(target, ["mkdir", "-p", skills_dir])
|
||||
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {target}:{dst}")
|
||||
# Wipe any prior copy so re-runs don't accumulate.
|
||||
_smolvm.machine_exec(target, ["rm", "-rf", dst])
|
||||
_smolvm.machine_cp(src, f"{target}:{dst}")
|
||||
_smolvm.machine_exec(target, ["chown", "-R", "node:node", dst])
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Supervise sidecar provisioning inside a running smolmachines
|
||||
bottle (PRD 0023 chunk 4d; PRD 0013 supervise plane).
|
||||
|
||||
Registers the per-bottle supervise sidecar as an HTTP MCP server
|
||||
in the agent's claude-code config so the agent discovers the
|
||||
stuck-recovery MCP tools (pipelock-block, capability-block) at
|
||||
startup.
|
||||
|
||||
Mirrors `backend.docker.provision.supervise` — same `claude mcp
|
||||
add` call, just dispatched via `smolvm machine exec` instead of
|
||||
`docker exec`, and against `<bundle_ip>:<port>` instead of the
|
||||
short `supervise` alias (no DNS in the TSI-allowlisted guest)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ....log import info, warn
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def provision_supervise(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Run `claude mcp add` inside the guest to register the
|
||||
supervise sidecar in claude-code's user config. No-op when
|
||||
bottle.supervise is False.
|
||||
|
||||
The URL is the agent-side endpoint launch.py populated after
|
||||
bundle bringup — `http://127.0.0.1:<host port>/` rather than
|
||||
the bundle's docker bridge IP, because that bridge isn't
|
||||
reachable from the smolvm guest on macOS.
|
||||
|
||||
Failure is logged but not fatal: the bottle still works (you
|
||||
just can't call supervise tools from the agent until the entry
|
||||
is added manually). The operator sees the warning at launch."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
url = plan.agent_supervise_url
|
||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||
# `claude mcp add --scope user` writes to ~/.claude.json. The
|
||||
# agent is the `node` user; smolvm machine_exec runs as root
|
||||
# by default, so we have to switch user explicitly and set
|
||||
# HOME so the config lands in /home/node/.claude.json (where
|
||||
# the agent's claude actually reads it from).
|
||||
r = _smolvm.machine_exec(
|
||||
target,
|
||||
[
|
||||
"runuser", "-u", "node", "--",
|
||||
"env", "HOME=/home/node",
|
||||
"claude", "mcp", "add",
|
||||
"--scope", "user",
|
||||
"--transport", "http",
|
||||
_SUPERVISE_MCP_NAME,
|
||||
url,
|
||||
],
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {url}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["provision_supervise"]
|
||||
@@ -1,32 +0,0 @@
|
||||
"""Copy the operator workspace into a smolmachines guest."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
|
||||
from ....log import info
|
||||
from ... import Bottle
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
"""Copy host cwd contents to the planned guest workspace."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_contents):
|
||||
return
|
||||
|
||||
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
||||
guest_path_q = shlex.quote(workspace.guest_path)
|
||||
guest_parent_q = shlex.quote(guest_parent)
|
||||
owner_q = shlex.quote(workspace.owner)
|
||||
mode_q = shlex.quote(workspace.mode)
|
||||
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
||||
bottle.exec(
|
||||
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
|
||||
user="root",
|
||||
)
|
||||
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
||||
bottle.exec(
|
||||
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
|
||||
user="root",
|
||||
)
|
||||
@@ -27,13 +27,11 @@ from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Mapping, Sequence
|
||||
|
||||
|
||||
|
||||
_SMOLVM = "smolvm"
|
||||
|
||||
|
||||
@@ -199,34 +197,6 @@ def machine_exec(
|
||||
)
|
||||
|
||||
|
||||
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
||||
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
|
||||
|
||||
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
|
||||
channel needs a brief warm-up before back-to-back exec calls are
|
||||
safe. Polling exits as soon as the channel is ready and fails
|
||||
loudly if the VM never responds."""
|
||||
deadline = time.monotonic() + timeout
|
||||
delay = 0.1
|
||||
while time.monotonic() < deadline:
|
||||
r = machine_exec(name, ["true"])
|
||||
if r.returncode == 0:
|
||||
return
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
break
|
||||
time.sleep(min(delay, remaining))
|
||||
delay = min(delay * 2, 0.5)
|
||||
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
|
||||
raise SmolvmError(
|
||||
argv,
|
||||
subprocess.CompletedProcess(
|
||||
args=argv, returncode=-1, stdout="",
|
||||
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def machine_cp(src: str, dst: str) -> None:
|
||||
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
||||
reference a path inside the VM, bare path for the host. Both
|
||||
|
||||
@@ -1,35 +1,34 @@
|
||||
"""Main CLI dispatcher.
|
||||
|
||||
Commands: cleanup, edit, info, init, list, resume, start, supervise
|
||||
Commands: cleanup, dashboard, edit, info, init, list, resume, start
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from ..log import Die, die, error
|
||||
from ..manifest import ManifestError
|
||||
from ..log import Die, die
|
||||
from ._common import PROG
|
||||
from . import list as _list_mod
|
||||
from .cleanup import cmd_cleanup
|
||||
from .dashboard import cmd_dashboard
|
||||
from .edit import cmd_edit
|
||||
from .info import cmd_info
|
||||
from .init import cmd_init
|
||||
from .resume import cmd_resume
|
||||
from .start import cmd_start
|
||||
from .supervise import cmd_supervise
|
||||
|
||||
cmd_list = _list_mod.cmd_list
|
||||
|
||||
COMMANDS = {
|
||||
"cleanup": cmd_cleanup,
|
||||
"dashboard": cmd_dashboard,
|
||||
"edit": cmd_edit,
|
||||
"info": cmd_info,
|
||||
"init": cmd_init,
|
||||
"list": cmd_list,
|
||||
"resume": cmd_resume,
|
||||
"start": cmd_start,
|
||||
"supervise": cmd_supervise,
|
||||
}
|
||||
|
||||
|
||||
@@ -37,13 +36,13 @@ def usage() -> None:
|
||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||
sys.stderr.write("Commands:\n")
|
||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||
sys.stderr.write(" dashboard view + approve/modify/reject pending supervise proposals (PRD 0013)\n")
|
||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
||||
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
||||
sys.stderr.write(" list list available agents or active containers\n")
|
||||
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
|
||||
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n")
|
||||
sys.stderr.write(" supervise view + approve/modify/reject pending supervise proposals (PRD 0013)\n\n")
|
||||
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n\n")
|
||||
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
||||
|
||||
|
||||
@@ -64,11 +63,6 @@ def main(argv: list[str] | None = None) -> int:
|
||||
die(f"unknown command: {command}")
|
||||
try:
|
||||
return handler(rest) or 0
|
||||
except ManifestError as e:
|
||||
# Manifest/config problems surface as a catchable exception;
|
||||
# print the reason and exit non-zero (same UX die() used to give).
|
||||
error(str(e))
|
||||
return 1
|
||||
except Die as e:
|
||||
return e.code if isinstance(e.code, int) else 1
|
||||
except KeyboardInterrupt:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -31,9 +31,6 @@ def cmd_info(argv: list[str]) -> int:
|
||||
f"first line: {prompt_first_line or '(empty)'}"
|
||||
)
|
||||
info(f"bottle : {agent.bottle}")
|
||||
identity = manifest.git_identity_summary(args.name)
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
if bottle.git:
|
||||
for e in bottle.git:
|
||||
info(
|
||||
|
||||
@@ -52,10 +52,8 @@ def cmd_resume(argv: list[str]) -> int:
|
||||
user_cwd=metadata.cwd or USER_CWD,
|
||||
identity=metadata.identity,
|
||||
)
|
||||
backend_name = metadata.backend or None
|
||||
return _launch_bottle(
|
||||
spec,
|
||||
dry_run=args.dry_run,
|
||||
remote_control=args.remote_control,
|
||||
backend_name=backend_name,
|
||||
)
|
||||
|
||||
+25
-11
@@ -2,8 +2,10 @@
|
||||
interactive claude-code session. The container is torn down when the
|
||||
session ends.
|
||||
|
||||
The launch core is shared with `cli.py resume <identity>` through
|
||||
the private orchestrator `_launch_bottle`.
|
||||
The launch core is shared with `cli.py resume <identity>` and (PRD
|
||||
0020 chunk 1+) the dashboard's in-process start flow: see the
|
||||
public helpers `prepare_with_preflight`, `attach_agent`, and the
|
||||
private orchestrator `_launch_bottle`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -69,7 +71,7 @@ def cmd_start(argv: list[str]) -> int:
|
||||
)
|
||||
|
||||
|
||||
# --- Launch helpers ------------------------------------------------------
|
||||
# --- Public helpers shared with the dashboard (PRD 0020) -----------------
|
||||
|
||||
|
||||
def prepare_with_preflight(
|
||||
@@ -82,11 +84,14 @@ def prepare_with_preflight(
|
||||
backend_name: str | None = None,
|
||||
) -> tuple[DockerBottlePlan | None, str]:
|
||||
"""Run `backend.prepare`, render the preflight summary via the
|
||||
injected callable, prompt y/N via the injected callable.
|
||||
injected callable, prompt y/N via the injected callable. The CLI
|
||||
binds these to stderr/stdin; the dashboard binds them to a
|
||||
curses modal.
|
||||
|
||||
`backend_name` selects which backend prepares the plan
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes
|
||||
whatever `--backend` resolved to.
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). Dashboard
|
||||
passes the value from its new-agent backend-picker modal; the
|
||||
CLI passes whatever `--backend` resolved to.
|
||||
|
||||
Returns `(plan, identity)`. `plan` is None on dry-run or
|
||||
operator-N, but `identity` is set as soon as `backend.prepare`
|
||||
@@ -117,10 +122,16 @@ def attach_agent(
|
||||
agent process's exit code.
|
||||
|
||||
`resume=True` adds `--continue` so claude picks up its most
|
||||
recent session non-interactively (no session-picker prompt).
|
||||
First-attach paths (`./cli.py start`) leave it False.
|
||||
recent session non-interactively (no session-picker prompt) —
|
||||
the right shape for the dashboard's Enter re-attach (PRD 0020
|
||||
chunk 3), where a bottle typically has exactly one session.
|
||||
First-attach paths (`./cli.py start`, the dashboard's new-agent
|
||||
flow) leave it False.
|
||||
|
||||
Used as the inner step of `./cli.py start`."""
|
||||
Used as the inner step of `./cli.py start` (one-shot) and by the
|
||||
dashboard, which calls it from inside a `curses.endwin → … →
|
||||
stdscr.refresh()` handoff so the curses surface gets out of the
|
||||
terminal's way while the agent has it."""
|
||||
runtime = runtime_for(agent_provider_template)
|
||||
info(
|
||||
f"attaching interactive {agent_provider_template} session "
|
||||
@@ -137,7 +148,8 @@ def attach_agent(
|
||||
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||
"""Inside the launch context, while the container is still
|
||||
alive: snapshot the transcript and mark for preservation if
|
||||
claude crashed."""
|
||||
claude crashed. Public for the dashboard's death-handling path
|
||||
(PRD 0020 open question 3)."""
|
||||
# FIXME: this captures Claude-specific session state. A follow-up
|
||||
# spike should explore freezing provider-neutral container state
|
||||
# instead of relying on each agent's transcript layout.
|
||||
@@ -150,7 +162,9 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||
|
||||
def settle_state(identity: str) -> None:
|
||||
"""Post-teardown housekeeping: print the resume hint if the
|
||||
state was preserved, otherwise reap the per-bottle state dir."""
|
||||
state was preserved, otherwise reap the per-bottle state dir.
|
||||
Public so the dashboard's explicit-stop path calls the same
|
||||
settlement the CLI uses on context exit."""
|
||||
if not identity:
|
||||
return
|
||||
if is_preserved(identity):
|
||||
|
||||
@@ -1,577 +0,0 @@
|
||||
"""supervise: list pending supervise proposals across all bottles and
|
||||
act on them (approve / modify / reject).
|
||||
|
||||
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
|
||||
approval handlers wire to the per-tool remediation engines:
|
||||
PRD 0014 (egress, retargeted from cred-proxy in PRD 0017
|
||||
chunk 3) writes routes.yaml + SIGHUPs egress; PRD 0015
|
||||
(pipelock) writes the allowlist + restarts pipelock; PRD 0016
|
||||
(capability) rebuilds the bottle Dockerfile.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import curses
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import traceback
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from .. import supervise as _supervise
|
||||
from ..backend.docker.bottle_state import read_metadata
|
||||
from ..backend.docker.capability_apply import (
|
||||
CapabilityApplyError,
|
||||
apply_capability_change,
|
||||
)
|
||||
from ..backend.docker.egress_apply import EgressApplyError, add_route
|
||||
from ..backend.docker.pipelock_apply import (
|
||||
PipelockApplyError,
|
||||
apply_allowlist_change,
|
||||
fetch_current_allowlist,
|
||||
parse_allowlist_content,
|
||||
render_allowlist_content,
|
||||
)
|
||||
from ..log import Die, error, info
|
||||
from ..supervise import (
|
||||
COMPONENT_FOR_TOOL,
|
||||
AuditEntry,
|
||||
Proposal,
|
||||
Response,
|
||||
STATUS_APPROVED,
|
||||
STATUS_MODIFIED,
|
||||
STATUS_REJECTED,
|
||||
TOOL_CAPABILITY_BLOCK,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_PIPELOCK_BLOCK,
|
||||
archive_proposal,
|
||||
list_pending_proposals,
|
||||
render_diff,
|
||||
write_audit_entry,
|
||||
write_response,
|
||||
)
|
||||
from ._common import PROG
|
||||
|
||||
|
||||
_REFRESH_INTERVAL_MS = 1000
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QueuedProposal:
|
||||
"""A pending proposal plus the queue dir it was found in."""
|
||||
|
||||
proposal: Proposal
|
||||
queue_dir: Path
|
||||
|
||||
|
||||
# Errors any remediation engine may raise. Caught by the TUI key
|
||||
# handlers and surfaced in the status line so a failed apply keeps
|
||||
# the proposal pending rather than crashing curses.
|
||||
ApplyError = (EgressApplyError, PipelockApplyError, CapabilityApplyError)
|
||||
|
||||
|
||||
def discover_pending() -> list[QueuedProposal]:
|
||||
"""Walk ~/.bot-bottle/queue/* and collect pending proposals."""
|
||||
queue_root = _supervise.bot_bottle_root() / "queue"
|
||||
if not queue_root.is_dir():
|
||||
return []
|
||||
out: list[QueuedProposal] = []
|
||||
for slug_dir in sorted(queue_root.iterdir()):
|
||||
if not slug_dir.is_dir():
|
||||
continue
|
||||
for proposal in list_pending_proposals(slug_dir):
|
||||
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
|
||||
out.sort(key=lambda q: q.proposal.arrival_timestamp)
|
||||
return out
|
||||
|
||||
|
||||
def _approval_status(qp: QueuedProposal, verb: str) -> str:
|
||||
"""Status-line text after a successful approval."""
|
||||
base = f"{verb} {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
|
||||
return base
|
||||
|
||||
|
||||
def _detail_lines(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
green_attr: int = 0,
|
||||
) -> list[tuple[str, int]]:
|
||||
"""Return the detail-view body as (text, curses-attr) tuples."""
|
||||
p = qp.proposal
|
||||
out: list[tuple[str, int]] = [
|
||||
(f"bottle: {p.bottle_slug}", 0),
|
||||
(f"tool: {p.tool}", 0),
|
||||
(f"id: {p.id}", 0),
|
||||
(f"arrived: {p.arrival_timestamp}", 0),
|
||||
(f"queue: {qp.queue_dir}", 0),
|
||||
("", 0),
|
||||
("justification:", 0),
|
||||
]
|
||||
out.extend((" " + line, 0) for line in p.justification.splitlines() or [""])
|
||||
out.extend([
|
||||
("", 0),
|
||||
(_proposed_payload_label(p.tool) + ":", 0),
|
||||
])
|
||||
out.extend((line, 0) for line in p.proposed_file.splitlines() or [""])
|
||||
if p.tool == TOOL_PIPELOCK_BLOCK:
|
||||
host = _failed_url_host(p.proposed_file)
|
||||
if host:
|
||||
out.append(("", 0))
|
||||
out.append((host, green_attr))
|
||||
return out
|
||||
|
||||
|
||||
def _failed_url_host(url: str) -> str:
|
||||
"""Best-effort hostname extraction from a pipelock-block proposal."""
|
||||
import urllib.parse
|
||||
|
||||
try:
|
||||
return urllib.parse.urlsplit(url.strip()).hostname or ""
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
|
||||
def _proposed_payload_label(tool: str) -> str:
|
||||
if tool == TOOL_PIPELOCK_BLOCK:
|
||||
return "failed URL"
|
||||
return "proposed file"
|
||||
|
||||
|
||||
def _suffix_for_tool(tool: str) -> str:
|
||||
if tool == TOOL_CAPABILITY_BLOCK:
|
||||
return ".dockerfile"
|
||||
return ".txt"
|
||||
|
||||
|
||||
# --- Operator actions ------------------------------------------------------
|
||||
|
||||
|
||||
def approve(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
notes: str = "",
|
||||
final_file: str | None = None,
|
||||
) -> None:
|
||||
"""Apply the proposal, write the waiting response, and audit it."""
|
||||
status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED
|
||||
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
|
||||
|
||||
diff_before, diff_after = "", ""
|
||||
if qp.proposal.tool == TOOL_EGRESS_BLOCK:
|
||||
diff_before, diff_after = add_route(
|
||||
qp.proposal.bottle_slug, file_to_apply,
|
||||
)
|
||||
elif qp.proposal.tool == TOOL_PIPELOCK_BLOCK:
|
||||
diff_before, diff_after = _apply_pipelock_url(
|
||||
qp.proposal.bottle_slug, file_to_apply,
|
||||
)
|
||||
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
_meta = read_metadata(qp.proposal.bottle_slug)
|
||||
if _meta is not None and not _meta.compose_project:
|
||||
raise CapabilityApplyError(
|
||||
"capability-block remediation is not supported for smolmachines "
|
||||
"bottles. Reject this proposal or handle the capability change "
|
||||
"manually, then restart the bottle."
|
||||
)
|
||||
diff_before, diff_after = apply_capability_change(
|
||||
qp.proposal.bottle_slug, file_to_apply,
|
||||
)
|
||||
|
||||
response = Response(
|
||||
proposal_id=qp.proposal.id,
|
||||
status=status,
|
||||
notes=notes,
|
||||
final_file=final_file,
|
||||
)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(
|
||||
qp, action=status, notes=notes,
|
||||
diff_before=diff_before, diff_after=diff_after,
|
||||
)
|
||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
archive_proposal(qp.queue_dir, qp.proposal.id)
|
||||
|
||||
|
||||
def reject(qp: QueuedProposal, *, reason: str) -> None:
|
||||
"""Write a rejection response and an audit entry."""
|
||||
response = Response(
|
||||
proposal_id=qp.proposal.id,
|
||||
status=STATUS_REJECTED,
|
||||
notes=reason,
|
||||
final_file=None,
|
||||
)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
||||
|
||||
|
||||
def _apply_pipelock_url(slug: str, failed_url: str) -> tuple[str, str]:
|
||||
"""Merge a pipelock-block failed URL's host into the allowlist."""
|
||||
import urllib.parse
|
||||
|
||||
parsed = urllib.parse.urlsplit(failed_url.strip())
|
||||
host = parsed.hostname or ""
|
||||
if not host:
|
||||
raise PipelockApplyError(
|
||||
f"proposed failed_url has no extractable host: {failed_url!r}"
|
||||
)
|
||||
current = fetch_current_allowlist(slug)
|
||||
hosts = parse_allowlist_content(current)
|
||||
if host not in hosts:
|
||||
hosts.append(host)
|
||||
return apply_allowlist_change(slug, render_allowlist_content(hosts))
|
||||
|
||||
|
||||
def _write_audit(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
action: str,
|
||||
notes: str,
|
||||
diff_before: str,
|
||||
diff_after: str,
|
||||
) -> None:
|
||||
"""Audit log for egress / pipelock tools."""
|
||||
component = COMPONENT_FOR_TOOL.get(qp.proposal.tool)
|
||||
if component is None:
|
||||
return
|
||||
write_audit_entry(AuditEntry(
|
||||
timestamp=datetime.now(timezone.utc).isoformat(),
|
||||
bottle_slug=qp.proposal.bottle_slug,
|
||||
component=component,
|
||||
operator_action=action,
|
||||
operator_notes=notes,
|
||||
justification=qp.proposal.justification,
|
||||
diff=render_diff(diff_before, diff_after, label=component),
|
||||
))
|
||||
|
||||
|
||||
# --- $EDITOR integration --------------------------------------------------
|
||||
|
||||
|
||||
def edit_in_editor(content: str, *, suffix: str = ".tmp") -> str | None:
|
||||
"""Open `content` in $EDITOR and return edited content, if changed."""
|
||||
editor = os.environ.get("EDITOR", "vim")
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w", suffix=suffix, delete=False, prefix="supervise-modify.",
|
||||
) as f:
|
||||
f.write(content)
|
||||
path = f.name
|
||||
try:
|
||||
subprocess.run([editor, path], check=False)
|
||||
with open(path) as f:
|
||||
edited = f.read()
|
||||
return edited if edited != content else None
|
||||
finally:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# --- TUI -------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_supervise(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} supervise", add_help=True)
|
||||
parser.add_argument(
|
||||
"--once", action="store_true",
|
||||
help="list pending proposals once and exit (no TUI)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.once:
|
||||
return _list_once()
|
||||
try:
|
||||
curses.wrapper(_main_loop)
|
||||
except KeyboardInterrupt:
|
||||
return 130
|
||||
except Die as e:
|
||||
if e.message:
|
||||
error(e.message)
|
||||
else:
|
||||
error("supervise exited on a fatal error (no detail captured).")
|
||||
return e.code if isinstance(e.code, int) else 1
|
||||
except Exception as e:
|
||||
log_path = _write_crash_log(e)
|
||||
error(f"supervise crashed: {type(e).__name__}: {e}")
|
||||
error(f"full traceback written to {log_path}")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def _write_crash_log(exc: BaseException) -> Path:
|
||||
"""Persist `exc`'s traceback to a stable file under ~/.bot-bottle/."""
|
||||
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
body = "".join(
|
||||
traceback.format_exception(type(exc), exc, exc.__traceback__)
|
||||
)
|
||||
entry = f"=== supervise crash {stamp} ===\n{body}\n"
|
||||
try:
|
||||
log_dir = _supervise.bot_bottle_root() / "logs"
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = log_dir / "supervise-crash.log"
|
||||
with path.open("a", encoding="utf-8") as fh:
|
||||
fh.write(entry)
|
||||
return path
|
||||
except OSError:
|
||||
fd, tmp = tempfile.mkstemp(
|
||||
prefix="bot-bottle-supervise-crash-", suffix=".log",
|
||||
)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(entry)
|
||||
return Path(tmp)
|
||||
|
||||
|
||||
def _list_once() -> int:
|
||||
pending = discover_pending()
|
||||
if not pending:
|
||||
info("no pending proposals")
|
||||
return 0
|
||||
for qp in pending:
|
||||
sys.stdout.write(
|
||||
f"{qp.proposal.arrival_timestamp} "
|
||||
f"[{qp.proposal.bottle_slug}] "
|
||||
f"{qp.proposal.tool} "
|
||||
f"{qp.proposal.id}\n"
|
||||
)
|
||||
sys.stdout.write(f" {qp.proposal.justification}\n")
|
||||
return 0
|
||||
|
||||
|
||||
def _try_init_green() -> int:
|
||||
"""Initialise a green color pair and return its attr, or 0."""
|
||||
try:
|
||||
curses.start_color()
|
||||
curses.use_default_colors()
|
||||
curses.init_pair(1, curses.COLOR_GREEN, -1)
|
||||
return curses.color_pair(1)
|
||||
except curses.error:
|
||||
return 0
|
||||
|
||||
|
||||
def _main_loop(stdscr: "curses._CursesWindow") -> None:
|
||||
curses.curs_set(0)
|
||||
stdscr.timeout(_REFRESH_INTERVAL_MS)
|
||||
green_attr = _try_init_green()
|
||||
selected = 0
|
||||
status_line = ""
|
||||
seen_ids: set[str] = set()
|
||||
|
||||
while True:
|
||||
pending = discover_pending()
|
||||
if selected >= len(pending):
|
||||
selected = max(0, len(pending) - 1)
|
||||
|
||||
live_ids = {qp.proposal.id for qp in pending}
|
||||
newly_arrived = live_ids - seen_ids
|
||||
if seen_ids and newly_arrived:
|
||||
try:
|
||||
curses.beep()
|
||||
except curses.error:
|
||||
pass
|
||||
for i, qp in enumerate(pending):
|
||||
if qp.proposal.id in newly_arrived:
|
||||
selected = i
|
||||
break
|
||||
seen_ids = live_ids
|
||||
|
||||
_render(
|
||||
stdscr, pending, selected, status_line,
|
||||
green_attr=green_attr,
|
||||
)
|
||||
|
||||
try:
|
||||
key = stdscr.getch()
|
||||
except KeyboardInterrupt:
|
||||
return
|
||||
|
||||
if key == -1:
|
||||
continue
|
||||
|
||||
status_line = ""
|
||||
|
||||
if key in (ord("q"), 27):
|
||||
return
|
||||
|
||||
if not pending:
|
||||
continue
|
||||
qp = pending[selected]
|
||||
|
||||
if key in (curses.KEY_DOWN, ord("j")):
|
||||
selected = min(selected + 1, len(pending) - 1)
|
||||
elif key in (curses.KEY_UP, ord("k")):
|
||||
selected = max(selected - 1, 0)
|
||||
elif key in (curses.KEY_ENTER, 10, 13):
|
||||
_detail_view(stdscr, qp, green_attr=green_attr)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
status_line = _approval_status(qp, "approved")
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("m"):
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is None:
|
||||
status_line = "modify aborted (no change)"
|
||||
else:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
status_line = _approval_status(qp, "modified+approved")
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("r"):
|
||||
reason = _prompt(stdscr, "reject reason: ")
|
||||
if reason:
|
||||
reject(qp, reason=reason)
|
||||
status_line = f"rejected {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||
else:
|
||||
status_line = "reject aborted (empty reason)"
|
||||
|
||||
|
||||
def _render(
|
||||
stdscr: "curses._CursesWindow",
|
||||
pending: list[QueuedProposal],
|
||||
selected: int,
|
||||
status_line: str,
|
||||
*,
|
||||
green_attr: int = 0,
|
||||
) -> None:
|
||||
stdscr.erase()
|
||||
h, w = stdscr.getmaxyx()
|
||||
header = f"bot-bottle supervise ({len(pending)} pending)"
|
||||
stdscr.addnstr(0, 0, header, w - 1, curses.A_BOLD)
|
||||
stdscr.hline(1, 0, curses.ACS_HLINE, w)
|
||||
|
||||
row = 2
|
||||
if not pending:
|
||||
stdscr.addnstr(
|
||||
row, 2,
|
||||
"no pending proposals; agents will queue here when they call a "
|
||||
"supervise tool",
|
||||
w - 4,
|
||||
)
|
||||
else:
|
||||
for i, qp in enumerate(pending):
|
||||
if row >= h - 3:
|
||||
break
|
||||
p = qp.proposal
|
||||
ts_short = (
|
||||
p.arrival_timestamp.split("T", 1)[1][:8]
|
||||
if "T" in p.arrival_timestamp else p.arrival_timestamp
|
||||
)
|
||||
cursor = "> " if i == selected else " "
|
||||
line = (
|
||||
f"{cursor}{ts_short} "
|
||||
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]} "
|
||||
f"{_proposed_payload_label(p.tool)}"
|
||||
)
|
||||
attr = curses.A_REVERSE if i == selected else curses.A_NORMAL
|
||||
stdscr.addnstr(row, 0, line, w - 1, attr)
|
||||
row += 1
|
||||
if row >= h - 3:
|
||||
break
|
||||
if p.justification:
|
||||
stdscr.addnstr(row, 4, p.justification[: max(0, w - 5)], w - 5)
|
||||
row += 1
|
||||
|
||||
footer = "[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit"
|
||||
stdscr.hline(h - 2, 0, curses.ACS_HLINE, w)
|
||||
stdscr.addnstr(h - 1, 0, footer, w - 1, curses.A_DIM)
|
||||
if status_line:
|
||||
stdscr.addnstr(h - 3, 0, status_line, w - 1, curses.A_BOLD)
|
||||
stdscr.refresh()
|
||||
|
||||
|
||||
def _detail_view(
|
||||
stdscr: "curses._CursesWindow",
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
green_attr: int = 0,
|
||||
) -> None:
|
||||
"""Render the full proposal. Scrollable. Press q to return."""
|
||||
lines = _detail_lines(qp, green_attr=green_attr)
|
||||
offset = 0
|
||||
while True:
|
||||
stdscr.erase()
|
||||
h, w = stdscr.getmaxyx()
|
||||
for i, (text, attr) in enumerate(lines[offset:offset + h - 1]):
|
||||
stdscr.addnstr(i, 0, text, w - 1, attr)
|
||||
stdscr.addnstr(
|
||||
h - 1, 0,
|
||||
"[j/k] scroll [g/G] top/bottom [a] approve [m] modify [r] reject [q] back",
|
||||
w - 1, curses.A_DIM,
|
||||
)
|
||||
stdscr.refresh()
|
||||
key = stdscr.getch()
|
||||
if key in (ord("q"), 27):
|
||||
return
|
||||
if key in (curses.KEY_DOWN, ord("j")):
|
||||
offset = min(offset + 1, max(0, len(lines) - 1))
|
||||
elif key in (curses.KEY_UP, ord("k")):
|
||||
offset = max(offset - 1, 0)
|
||||
elif key == ord("g"):
|
||||
offset = 0
|
||||
elif key == ord("G"):
|
||||
offset = max(0, len(lines) - 1)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
elif key == ord("m"):
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is not None:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
elif key == ord("r"):
|
||||
reason = _prompt(stdscr, "reject reason: ")
|
||||
if reason:
|
||||
reject(qp, reason=reason)
|
||||
return
|
||||
|
||||
|
||||
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
|
||||
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
|
||||
suffix = _suffix_for_tool(qp.proposal.tool)
|
||||
curses.endwin()
|
||||
try:
|
||||
edited = edit_in_editor(qp.proposal.proposed_file, suffix=suffix)
|
||||
finally:
|
||||
stdscr.refresh()
|
||||
return edited
|
||||
|
||||
|
||||
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str:
|
||||
"""One-line input at the bottom of the screen."""
|
||||
curses.curs_set(1)
|
||||
h, _ = stdscr.getmaxyx()
|
||||
stdscr.move(h - 2, 0)
|
||||
stdscr.clrtoeol()
|
||||
stdscr.addstr(h - 2, 0, label)
|
||||
stdscr.refresh()
|
||||
curses.echo()
|
||||
try:
|
||||
raw = stdscr.getstr(h - 2, len(label), 200)
|
||||
finally:
|
||||
curses.noecho()
|
||||
curses.curs_set(0)
|
||||
return raw.decode("utf-8", errors="replace").strip()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"QueuedProposal",
|
||||
"approve",
|
||||
"cmd_supervise",
|
||||
"discover_pending",
|
||||
"edit_in_editor",
|
||||
"reject",
|
||||
]
|
||||
@@ -1,325 +0,0 @@
|
||||
"""Host Codex auth helpers.
|
||||
|
||||
Reads the host's Codex ChatGPT/device-login auth state and returns only
|
||||
the short-lived access token needed by egress. This module deliberately
|
||||
does not expose refresh tokens or raw auth payloads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from .log import die
|
||||
from .util import expand_tilde
|
||||
|
||||
|
||||
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
|
||||
env = os.environ if host_env is None else host_env
|
||||
home = env.get("CODEX_HOME")
|
||||
if home:
|
||||
return Path(expand_tilde(home)) / "auth.json"
|
||||
return Path.home() / ".codex" / "auth.json"
|
||||
|
||||
|
||||
def codex_host_access_token(
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> str:
|
||||
path = codex_auth_path(host_env)
|
||||
if not path.is_file():
|
||||
die(
|
||||
f"codex host credentials: auth file missing at {path}. "
|
||||
"Run `codex login --device-auth` on the host or disable "
|
||||
"agent_provider.forward_host_credentials."
|
||||
)
|
||||
raw = _read_auth_object(path)
|
||||
|
||||
auth_mode = raw.get("auth_mode")
|
||||
if not isinstance(auth_mode, str) or auth_mode == "api_key":
|
||||
die(
|
||||
"codex host credentials: host Codex auth is not user/device "
|
||||
"auth. Run `codex login --device-auth` on the host."
|
||||
)
|
||||
|
||||
tokens = raw.get("tokens")
|
||||
if not isinstance(tokens, dict):
|
||||
die(f"codex host credentials: {path} is missing tokens")
|
||||
access = tokens.get("access_token")
|
||||
if not isinstance(access, str) or not access:
|
||||
die(
|
||||
f"codex host credentials: {path} is missing tokens.access_token. "
|
||||
"Run `codex login --device-auth` on the host."
|
||||
)
|
||||
|
||||
exp = _jwt_exp(access)
|
||||
if exp is None:
|
||||
die("codex host credentials: tokens.access_token is not a JWT with exp")
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
if exp <= check_now:
|
||||
die(
|
||||
"codex host credentials: host Codex access token is expired. "
|
||||
"Run `codex login --device-auth` on the host and restart the bottle."
|
||||
)
|
||||
return access
|
||||
|
||||
|
||||
def codex_dummy_auth_json(
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> str:
|
||||
"""Return a non-secret `auth.json` that keeps Codex in the host's
|
||||
auth branch while egress owns the real bearer token.
|
||||
|
||||
The dummy access/id tokens carry the *host* token's real `exp` so
|
||||
Codex's proactive refresh lifecycle (it refreshes when its local
|
||||
access token is at/past expiry) tracks the real token instead of
|
||||
firing after an artificial TTL. Codex cannot refresh inside the
|
||||
bottle — the refresh token is a placeholder and the OpenAI token
|
||||
endpoint is off-route — so a shorter dummy exp would drop Codex to
|
||||
the sign-in screen the moment it lapsed, even while egress still
|
||||
holds a valid bearer."""
|
||||
path = codex_auth_path(host_env)
|
||||
access = codex_host_access_token(host_env, now=now)
|
||||
raw = _read_auth_object(path)
|
||||
host_exp = _jwt_exp(access)
|
||||
exp_ts = int(host_exp.timestamp()) if host_exp is not None else None
|
||||
dummy = _redact_codex_auth(deepcopy(raw), now=now, exp_ts=exp_ts)
|
||||
return json.dumps(dummy, indent=2, sort_keys=True) + "\n"
|
||||
|
||||
|
||||
def write_codex_dummy_auth_file(
|
||||
path: Path,
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(codex_dummy_auth_json(host_env, now=now))
|
||||
path.chmod(0o600)
|
||||
|
||||
|
||||
def _read_auth_object(path: Path) -> dict:
|
||||
try:
|
||||
raw = json.loads(path.read_text())
|
||||
except (OSError, json.JSONDecodeError) as e:
|
||||
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
|
||||
if not isinstance(raw, dict):
|
||||
die(f"codex host credentials: {path} must contain a JSON object")
|
||||
return raw
|
||||
|
||||
|
||||
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
||||
if exp_ts is not None:
|
||||
return exp_ts
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
return int(check_now.timestamp()) + 3600
|
||||
|
||||
|
||||
def _dummy_timestamp(now: datetime | None = None) -> str:
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
if check_now.tzinfo is None:
|
||||
check_now = check_now.replace(tzinfo=timezone.utc)
|
||||
check_now = check_now.astimezone(timezone.utc)
|
||||
return check_now.isoformat(timespec="milliseconds").replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
|
||||
return _encode_dummy_jwt({
|
||||
"exp": _dummy_exp(now, exp_ts),
|
||||
"sub": "bot-bottle-placeholder",
|
||||
})
|
||||
|
||||
|
||||
def _dummy_jwt_from_host(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> str:
|
||||
if not isinstance(value, str):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
parts = value.split(".")
|
||||
if len(parts) < 2:
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
try:
|
||||
payload = json.loads(_b64url_decode(parts[1]))
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
if not isinstance(payload, dict):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
return _encode_dummy_jwt(_redact_jwt_payload(payload, now=now, exp_ts=exp_ts))
|
||||
|
||||
|
||||
def _encode_dummy_jwt(payload: dict) -> str:
|
||||
def enc(obj: dict) -> str:
|
||||
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||
|
||||
return f"{enc({'alg': 'none', 'typ': 'JWT'})}.{enc(payload)}.placeholder"
|
||||
|
||||
|
||||
def _redact_jwt_payload(
|
||||
payload: dict,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
exp_ts: int | None = None,
|
||||
) -> dict:
|
||||
out = _redact_claims(payload)
|
||||
if not isinstance(out, dict):
|
||||
out = {}
|
||||
out["exp"] = _dummy_exp(now, exp_ts)
|
||||
out.setdefault("sub", "bot-bottle-placeholder")
|
||||
return out
|
||||
|
||||
|
||||
def _redact_claims(value: object) -> object:
|
||||
if isinstance(value, dict):
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in value.items():
|
||||
lower = key.lower()
|
||||
if key == "https://api.openai.com/profile":
|
||||
out[key] = _redact_profile_claim(inner)
|
||||
elif key == "https://api.openai.com/auth":
|
||||
out[key] = _redact_auth_claim(inner)
|
||||
elif lower == "email":
|
||||
out[key] = "bot-bottle@example.invalid"
|
||||
elif lower == "email_verified":
|
||||
out[key] = True
|
||||
elif lower in {"exp", "iat", "nbf", "auth_time", "pwd_auth_time"}:
|
||||
out[key] = inner if isinstance(inner, (int, float)) else 0
|
||||
elif lower in {"aud", "scp", "amr"}:
|
||||
out[key] = inner if isinstance(inner, list) else []
|
||||
elif isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, dict):
|
||||
out[key] = {}
|
||||
elif isinstance(inner, list):
|
||||
out[key] = []
|
||||
else:
|
||||
out[key] = "bot-bottle-placeholder"
|
||||
return out
|
||||
if isinstance(value, list):
|
||||
return []
|
||||
return "bot-bottle-placeholder"
|
||||
|
||||
|
||||
def _redact_profile_claim(value: object) -> dict:
|
||||
profile = value if isinstance(value, dict) else {}
|
||||
return {
|
||||
"email": "bot-bottle@example.invalid",
|
||||
"email_verified": bool(profile.get("email_verified", True)),
|
||||
}
|
||||
|
||||
|
||||
def _redact_auth_claim(value: object) -> dict:
|
||||
auth = value if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in auth.items():
|
||||
lower = key.lower()
|
||||
if lower == "chatgpt_plan_type" and isinstance(inner, str) and inner:
|
||||
out[key] = inner
|
||||
elif lower == "chatgpt_account_id" and isinstance(inner, str) and inner:
|
||||
# Current Codex uses the selected account id when building
|
||||
# ChatGPT requests. Keep that non-secret identifier aligned
|
||||
# with the host while egress owns the real bearer token.
|
||||
out[key] = inner
|
||||
elif lower == "localhost" and isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, list):
|
||||
out[key] = []
|
||||
elif isinstance(inner, dict):
|
||||
out[key] = {}
|
||||
else:
|
||||
out[key] = "bot-bottle-placeholder"
|
||||
out.setdefault("chatgpt_plan_type", "unknown")
|
||||
out.setdefault("user_id", "bot-bottle-placeholder")
|
||||
out.setdefault("chatgpt_user_id", "bot-bottle-placeholder")
|
||||
out.setdefault("chatgpt_account_id", "bot-bottle-placeholder")
|
||||
return out
|
||||
|
||||
|
||||
def _redact_codex_auth(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> object:
|
||||
auth = value if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in auth.items():
|
||||
lower = key.lower()
|
||||
if lower == "auth_mode" and isinstance(inner, str) and inner:
|
||||
out[key] = inner
|
||||
elif lower == "openai_api_key":
|
||||
out[key] = None
|
||||
elif lower == "last_refresh":
|
||||
# Codex parses this as a timestamp on startup. Keep the
|
||||
# schema valid without copying host-side session metadata.
|
||||
out[key] = _dummy_timestamp(now)
|
||||
elif lower == "tokens":
|
||||
out[key] = _redact_token_block(inner, now=now, exp_ts=exp_ts)
|
||||
else:
|
||||
out[key] = _redact_unknown_auth_value(inner)
|
||||
return out
|
||||
|
||||
|
||||
def _redact_token_block(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> dict[str, object]:
|
||||
tokens = value if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in tokens.items():
|
||||
lower = key.lower()
|
||||
if lower in {"access_token", "id_token"}:
|
||||
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
|
||||
elif lower == "account_id" and isinstance(inner, str) and inner:
|
||||
# Current Codex uses this non-secret selected account id
|
||||
# while egress owns the real bearer token.
|
||||
out[key] = inner
|
||||
else:
|
||||
out[key] = _redact_unknown_auth_value(inner)
|
||||
return out
|
||||
|
||||
|
||||
def _redact_unknown_auth_value(value: object) -> object:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return {}
|
||||
if isinstance(value, list):
|
||||
return []
|
||||
if value is None:
|
||||
return None
|
||||
return "bot-bottle-placeholder"
|
||||
|
||||
|
||||
def _jwt_exp(token: str) -> datetime | None:
|
||||
parts = token.split(".")
|
||||
if len(parts) < 2:
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(_b64url_decode(parts[1]))
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return None
|
||||
if not isinstance(payload, dict):
|
||||
return None
|
||||
exp = payload.get("exp")
|
||||
if not isinstance(exp, (int, float)):
|
||||
return None
|
||||
return datetime.fromtimestamp(exp, timezone.utc)
|
||||
|
||||
|
||||
def _b64url_decode(value: str) -> str:
|
||||
padded = value + ("=" * (-len(value) % 4))
|
||||
return base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"codex_auth_path",
|
||||
"codex_dummy_auth_json",
|
||||
"codex_host_access_token",
|
||||
"write_codex_dummy_auth_file",
|
||||
]
|
||||
@@ -1,226 +0,0 @@
|
||||
"""Claude agent provider plugin (PRD 0050, contrib).
|
||||
|
||||
The Claude-specific behavior previously inlined under
|
||||
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
||||
api.anthropic.com egress route, OAuth-token placeholder), plus
|
||||
the `claude mcp add` invocation that registers the supervise
|
||||
sidecar in claude-code's user config (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ...agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...egress import EgressRoute
|
||||
from ...log import die, info, warn
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
return f"{guest_home}/.claude/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="claude",
|
||||
command="claude",
|
||||
image="bot-bottle-claude:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||
prompt_mode="append_file",
|
||||
bypass_args=("--dangerously-skip-permissions",),
|
||||
resume_args=("--continue",),
|
||||
remote_control_args=("--remote-control",),
|
||||
)
|
||||
|
||||
|
||||
class ClaudeAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
) -> AgentProvisionPlan:
|
||||
del forward_host_credentials, host_env # Codex-only knobs
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
|
||||
"DISABLE_ERROR_REPORTING": "1",
|
||||
}
|
||||
claude_config = state_dir / "claude.json"
|
||||
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
|
||||
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
||||
claude_config.write_text(json.dumps({
|
||||
"hasCompletedOnboarding": True,
|
||||
"theme": "dark",
|
||||
"bypassPermissionsModeAccepted": True,
|
||||
"projects": claude_projects,
|
||||
}, indent=2) + "\n")
|
||||
claude_config.chmod(0o600)
|
||||
files = (
|
||||
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
|
||||
)
|
||||
egress_routes = (EgressRoute(
|
||||
host="api.anthropic.com",
|
||||
auth_scheme="Bearer" if auth_token else "",
|
||||
token_ref=auth_token,
|
||||
tls_passthrough=True,
|
||||
),)
|
||||
hidden_env_names: frozenset[str] = frozenset()
|
||||
if auth_token:
|
||||
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
files=files,
|
||||
egress_routes=egress_routes,
|
||||
hidden_env_names=hidden_env_names,
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||
on the host into the guest's claude-code skills dir. No-op
|
||||
when the agent has no skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||
Returns the in-guest path iff the agent has a non-empty
|
||||
prompt (drives `--append-system-prompt-file`); the file is
|
||||
copied either way so the path always exists."""
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path)
|
||||
bottle.exec(
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the claude-side declarative provision steps from
|
||||
`plan.agent_provision` — today that's the `claude.json`
|
||||
trust-marker file. Hot-replace this with a richer flow as
|
||||
claude-code's harness shape evolves."""
|
||||
provision = plan.agent_provision
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for command in provision.pre_copy:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
for command in provision.verify:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Run `claude mcp add` inside the agent guest to register the
|
||||
supervise sidecar in claude-code's user config (~/.claude.json).
|
||||
|
||||
Failure is logged but not fatal — the bottle still works without
|
||||
the entry; the operator can register it manually."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
info(f"registering supervise MCP server in agent claude config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"claude mcp add --scope user --transport http "
|
||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||
user="node",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {supervise_url}"
|
||||
)
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
@@ -1,271 +0,0 @@
|
||||
"""Codex agent provider plugin (PRD 0050, contrib).
|
||||
|
||||
The Codex-specific behavior previously inlined under
|
||||
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
||||
chatgpt.com / api.openai.com egress routes, optional host-credential
|
||||
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
||||
invocation that registers the supervise sidecar in Codex's
|
||||
~/.codex/config.toml (PRD 0050)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ...agent_provider import (
|
||||
CODEX_HOST_CREDENTIAL_HOSTS,
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionCommand,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
||||
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
||||
from ...log import die, info, warn
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
# Codex agents still read skills from the claude-code convention
|
||||
# (~/.claude/skills/) — the bot-bottle-codex image follows the
|
||||
# same layout. If Codex grows native skill discovery later,
|
||||
# change here.
|
||||
return f"{guest_home}/.claude/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="codex",
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
resume_args=("resume", "--last"),
|
||||
remote_control_args=(),
|
||||
)
|
||||
|
||||
|
||||
class CodexAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
) -> AgentProvisionPlan:
|
||||
del auth_token # Claude-only knob
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
"CODEX_CA_CERTIFICATE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
||||
if forward_host_credentials:
|
||||
env_vars["CODEX_HOME"] = auth_dir
|
||||
|
||||
dirs = [AgentProvisionDir(auth_dir)]
|
||||
files: list[AgentProvisionFile] = []
|
||||
pre_copy: list[AgentProvisionCommand] = []
|
||||
verify: list[AgentProvisionCommand] = []
|
||||
provisioned_env: dict[str, str] = {}
|
||||
|
||||
config_path = f"{auth_dir}/config.toml"
|
||||
config_file = state_dir / "codex-config.toml"
|
||||
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
||||
config_file.write_text(
|
||||
f'[projects."{toml_path}"]\n'
|
||||
'trust_level = "trusted"\n'
|
||||
)
|
||||
config_file.chmod(0o600)
|
||||
files.append(AgentProvisionFile(config_file, config_path))
|
||||
|
||||
egress_routes: list[EgressRoute] = []
|
||||
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
||||
egress_routes.append(EgressRoute(
|
||||
host=host,
|
||||
auth_scheme="Bearer" if forward_host_credentials else "",
|
||||
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
||||
tls_passthrough=True,
|
||||
))
|
||||
|
||||
if forward_host_credentials:
|
||||
_host_env = host_env or dict(os.environ)
|
||||
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = (
|
||||
codex_host_access_token(_host_env)
|
||||
)
|
||||
auth_file = state_dir / "codex-auth.json"
|
||||
write_codex_dummy_auth_file(auth_file, _host_env)
|
||||
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
||||
pre_copy.append(AgentProvisionCommand((
|
||||
"find", auth_dir,
|
||||
"-maxdepth", "1",
|
||||
"-type", "f",
|
||||
"(",
|
||||
"-name", "*.sqlite",
|
||||
"-o", "-name", "*.sqlite-*",
|
||||
"-o", "-name", "*.codex-repair-*.bak",
|
||||
")",
|
||||
"-delete",
|
||||
), "codex host credentials: could not reset runtime db files"))
|
||||
verify.append(AgentProvisionCommand((
|
||||
"runuser", "-u", "node", "--",
|
||||
"env",
|
||||
f"HOME={guest_home}",
|
||||
f"CODEX_HOME={auth_dir}",
|
||||
"codex", "login", "status",
|
||||
), (
|
||||
"codex host credentials: dummy auth was copied into the "
|
||||
"guest, but Codex did not accept it"
|
||||
)))
|
||||
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
dirs=tuple(dirs),
|
||||
files=tuple(files),
|
||||
pre_copy=tuple(pre_copy),
|
||||
verify=tuple(verify),
|
||||
egress_routes=tuple(egress_routes),
|
||||
provisioned_env=provisioned_env,
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||
on the host into the guest. No-op when the agent has no
|
||||
skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||
Codex reads it via the agent's `Read and follow the
|
||||
instructions in <path>.` bootstrap (see `prompt_args`); the
|
||||
file is copied either way so the path always exists."""
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path)
|
||||
bottle.exec(
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the codex-side declarative provision steps from
|
||||
`plan.agent_provision`: the `~/.codex/` dir + config.toml
|
||||
trust marker, plus the dummy-auth.json drop + `codex login
|
||||
status` verify when host-credential forwarding is on."""
|
||||
provision = plan.agent_provision
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for command in provision.pre_copy:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
for command in provision.verify:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Run `codex mcp add` inside the agent guest to register the
|
||||
supervise sidecar in Codex's user config (~/.codex/config.toml).
|
||||
|
||||
Mirrors the Claude provider's `claude mcp add` flow — failure
|
||||
is logged but not fatal."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"codex mcp add --transport http "
|
||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||
user="node",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`codex mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"codex mcp add --transport http supervise {supervise_url}"
|
||||
)
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
@@ -1,121 +0,0 @@
|
||||
"""Gitea deploy-key provisioner (PRD 0048, contrib).
|
||||
|
||||
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
|
||||
them using the Gitea deploy-key HTTP API. No new Python dependencies —
|
||||
only stdlib `urllib.request` and `subprocess`."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from ...deploy_key_provisioner import DeployKeyProvisioner
|
||||
|
||||
|
||||
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
|
||||
"""Manages deploy keys on a Gitea instance."""
|
||||
|
||||
def __init__(self, *, token: str, api_url: str) -> None:
|
||||
self._token = token
|
||||
self._api_url = api_url.rstrip("/")
|
||||
|
||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||
"""Generate an ed25519 keypair, register the public half as a
|
||||
repo deploy key, and return `(key_id, private_key_bytes)`.
|
||||
|
||||
The key is registered with `read_only=False` because git-gate
|
||||
needs push access to forward gitleaks-scanned refs upstream."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
key_path = Path(tmpdir) / "key"
|
||||
subprocess.run(
|
||||
[
|
||||
"ssh-keygen", "-t", "ed25519",
|
||||
"-f", str(key_path),
|
||||
"-N", "",
|
||||
],
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
private_key = key_path.read_bytes()
|
||||
public_key = key_path.with_suffix(".pub").read_text().strip()
|
||||
|
||||
owner, repo = _split_owner_repo(owner_repo)
|
||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys"
|
||||
payload = json.dumps({
|
||||
"key": public_key,
|
||||
"read_only": False,
|
||||
"title": title,
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=payload,
|
||||
headers={
|
||||
"Authorization": f"token {self._token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
body = json.loads(resp.read())
|
||||
except urllib.error.HTTPError as exc:
|
||||
_body = _read_error_body(exc)
|
||||
raise RuntimeError(
|
||||
f"failed to create deploy key for {owner_repo}: "
|
||||
f"HTTP {exc.code} — {_body}"
|
||||
) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise RuntimeError(
|
||||
f"failed to create deploy key for {owner_repo}: {exc.reason}"
|
||||
) from exc
|
||||
|
||||
return str(body["id"]), private_key
|
||||
|
||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||
"""Delete the deploy key. HTTP 404 (already gone) is success.
|
||||
All other errors raise RuntimeError so teardown halts."""
|
||||
owner, repo = _split_owner_repo(owner_repo)
|
||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys/{key_id}"
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={"Authorization": f"token {self._token}"},
|
||||
method="DELETE",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req):
|
||||
pass
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return
|
||||
_body = _read_error_body(exc)
|
||||
raise RuntimeError(
|
||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||
f"HTTP {exc.code} — {_body}"
|
||||
) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise RuntimeError(
|
||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||
f"{exc.reason}"
|
||||
) from exc
|
||||
|
||||
|
||||
def _split_owner_repo(owner_repo: str) -> tuple[str, str]:
|
||||
"""Split `'owner/repo'` into `('owner', 'repo')`."""
|
||||
parts = owner_repo.split("/", 1)
|
||||
if len(parts) != 2 or not all(parts):
|
||||
raise ValueError(
|
||||
f"expected 'owner/repo' format, got {owner_repo!r}"
|
||||
)
|
||||
return parts[0], parts[1]
|
||||
|
||||
|
||||
def _read_error_body(exc: urllib.error.HTTPError) -> str:
|
||||
try:
|
||||
return exc.read().decode("utf-8", errors="replace")
|
||||
except Exception:
|
||||
return ""
|
||||
@@ -1,52 +0,0 @@
|
||||
"""Deploy-key provisioner interface and factory (PRD 0048).
|
||||
|
||||
The core defines the abstract contract; concrete implementations live
|
||||
in `bot_bottle/contrib/<provider>/deploy_key_provisioner.py`. The
|
||||
factory `get_provisioner` imports contrib modules lazily so that a
|
||||
missing optional dependency in one provider doesn't break unrelated
|
||||
features."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class DeployKeyProvisioner(ABC):
|
||||
"""Manages a single deploy-key lifecycle on a remote forge."""
|
||||
|
||||
@abstractmethod
|
||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||
"""Generate a keypair and register the public half as a
|
||||
deploy key on the forge.
|
||||
|
||||
`owner_repo` is the `<owner>/<repo>` path (no `.git` suffix).
|
||||
`title` is the human-readable label shown in the forge UI.
|
||||
|
||||
Returns `(key_id, private_key_bytes)` where `key_id` is opaque
|
||||
to the caller and is only ever passed back to `delete`."""
|
||||
|
||||
@abstractmethod
|
||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||
"""Delete the registered deploy key.
|
||||
|
||||
Must not raise if the key is already absent (HTTP 404 is
|
||||
success). Must raise for all other failures so teardown halts."""
|
||||
|
||||
|
||||
def get_provisioner(
|
||||
provider: str, token: str, api_url: str
|
||||
) -> DeployKeyProvisioner:
|
||||
"""Instantiate the contrib provisioner for `provider`.
|
||||
|
||||
Raises `ManifestError` for unknown providers so the error surfaces
|
||||
at parse time rather than at runtime."""
|
||||
if provider == "gitea":
|
||||
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
||||
GiteaDeployKeyProvisioner,
|
||||
)
|
||||
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
||||
from .manifest_util import ManifestError
|
||||
raise ManifestError(
|
||||
f"unknown provisioned_key provider: {provider!r}; "
|
||||
f"available: gitea"
|
||||
)
|
||||
+86
-110
@@ -24,19 +24,12 @@ flow (PRD 0014) at egress and renames the MCP tool.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .egress_addon_core import Route
|
||||
from .log import die
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Bottle
|
||||
|
||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||
from .manifest import Bottle
|
||||
|
||||
|
||||
# DNS name agents will dial for the per-bottle egress sidecar.
|
||||
@@ -55,30 +48,32 @@ EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressRoute(Route):
|
||||
"""Host-side extension of the addon's `Route`.
|
||||
class EgressRoute:
|
||||
"""One resolved route on the egress sidecar.
|
||||
|
||||
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
|
||||
from `egress_addon_core.Route` — those are the fields that cross the
|
||||
YAML wire into the sidecar. The three fields below are host-only and
|
||||
are never serialised to the addon.
|
||||
`host` matches the request's hostname (case-insensitive). The
|
||||
optional `path_allowlist` constrains the URL path; empty tuple
|
||||
means no path-level filtering. The `auth_scheme` / `token_env` /
|
||||
`token_ref` triple is the credential-injection config; empty
|
||||
strings mean "no auth injection" (the manifest's nested `auth`
|
||||
block was omitted).
|
||||
|
||||
`token_ref` is the host env var the CLI reads at launch and forwards
|
||||
into the container's environ under `token_env`. Routes that share a
|
||||
`token_ref` coalesce to one `token_env` slot.
|
||||
`token_env` is the env-var slot inside the egress container
|
||||
(e.g. `EGRESS_TOKEN_0`); `token_ref` is the host env var
|
||||
the CLI reads at launch and forwards into the container's environ
|
||||
under `token_env`. Routes that share a `token_ref` coalesce to
|
||||
one `token_env` slot.
|
||||
|
||||
`roles` carries the manifest route's role tuple (reserved for
|
||||
future use; always empty today).
|
||||
|
||||
`tls_passthrough` signals that pipelock must not TLS-MITM this
|
||||
host — either because the manifest declared `pipelock.tls_passthrough:
|
||||
true` (lifted in `egress_manifest_routes`) or because a provider
|
||||
route set it (e.g. egress injects its own Bearer on that host
|
||||
after the agent boundary and pipelock's header DLP would block it)."""
|
||||
`roles` carries the manifest route's optional role markers (see
|
||||
`manifest.EGRESS_ROLES`). The launch step reads these for
|
||||
side effects like the claude-code OAuth placeholder env."""
|
||||
|
||||
host: str
|
||||
path_allowlist: tuple[str, ...] = ()
|
||||
auth_scheme: str = ""
|
||||
token_env: str = ""
|
||||
token_ref: str = ""
|
||||
roles: tuple[str, ...] = ()
|
||||
tls_passthrough: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -135,60 +130,55 @@ class EgressPlan:
|
||||
def egress_manifest_routes(
|
||||
bottle: Bottle,
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
|
||||
Order is preserved. Token slots are not assigned here — slot assignment
|
||||
is a final step in `egress_routes_for_bottle` after provider and manifest
|
||||
routes are merged."""
|
||||
"""Lift each `bottle.egress.routes[]` manifest entry into a
|
||||
resolved EgressRoute. Order is preserved so route lookup at
|
||||
the proxy is stable.
|
||||
|
||||
Token-env slots are assigned per distinct `token_ref`: the first
|
||||
authenticated route with `token_ref` "GH_PAT" gets
|
||||
`EGRESS_TOKEN_0`; a second route with the same `token_ref`
|
||||
shares slot 0. Unauthenticated routes (`auth` omitted) contribute
|
||||
no slot.
|
||||
|
||||
This is the effective set the addon enforces. Provider runtime
|
||||
routes are intentionally not injected implicitly; every allowed
|
||||
host must come from the home-owned bottle manifest."""
|
||||
out: list[EgressRoute] = []
|
||||
slot_for_token: dict[str, str] = {}
|
||||
for r in bottle.egress.routes:
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
path_allowlist=r.PathAllowlist,
|
||||
auth_scheme=r.AuthScheme,
|
||||
token_ref=r.TokenRef,
|
||||
roles=r.Role,
|
||||
tls_passthrough=r.Pipelock.TlsPassthrough,
|
||||
))
|
||||
if r.AuthScheme and r.TokenRef:
|
||||
token_env = slot_for_token.get(r.TokenRef)
|
||||
if token_env is None:
|
||||
token_env = f"EGRESS_TOKEN_{len(slot_for_token)}"
|
||||
slot_for_token[r.TokenRef] = token_env
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
path_allowlist=r.PathAllowlist,
|
||||
auth_scheme=r.AuthScheme,
|
||||
token_env=token_env,
|
||||
token_ref=r.TokenRef,
|
||||
roles=r.Role,
|
||||
))
|
||||
else:
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
path_allowlist=r.PathAllowlist,
|
||||
roles=r.Role,
|
||||
))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_routes_for_bottle(
|
||||
bottle: Bottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Effective egress routes for the agent.
|
||||
"""Effective egress routes. This is what gets rendered into
|
||||
routes.yaml + what the addon enforces.
|
||||
|
||||
Provider routes own their hosts outright; manifest routes for hosts
|
||||
not claimed by any provider are appended. Token slots are assigned
|
||||
in a final pass over the merged list in order, so provisioned routes
|
||||
get the lower slot numbers."""
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
merged = list(provider_routes) + [
|
||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||
]
|
||||
return _assign_token_slots(merged)
|
||||
|
||||
|
||||
def _assign_token_slots(
|
||||
routes: list[EgressRoute],
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
|
||||
|
||||
Routes sharing a token_ref share a slot. Unauthenticated routes
|
||||
(no auth_scheme / token_ref) keep token_env empty."""
|
||||
slot_for_ref: dict[str, str] = {}
|
||||
out: list[EgressRoute] = []
|
||||
for r in routes:
|
||||
if r.auth_scheme and r.token_ref:
|
||||
slot = slot_for_ref.get(r.token_ref)
|
||||
if slot is None:
|
||||
slot = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
||||
slot_for_ref[r.token_ref] = slot
|
||||
out.append(dataclasses.replace(r, token_env=slot))
|
||||
else:
|
||||
out.append(r)
|
||||
return tuple(out)
|
||||
Operators that want to allow a host declare it directly in
|
||||
`bottle.egress.routes` as an authenticated route or bare-pass entry
|
||||
(`- host: <name>`). The legacy `bottle.egress.allowlist`
|
||||
folding is gone — egress is the single allowlist surface."""
|
||||
return egress_manifest_routes(bottle)
|
||||
|
||||
|
||||
def egress_token_env_map(
|
||||
@@ -203,7 +193,7 @@ def egress_token_env_map(
|
||||
silently picking one."""
|
||||
out: dict[str, str] = {}
|
||||
for r in routes:
|
||||
if not (r.auth_scheme and r.token_ref and r.token_env):
|
||||
if not r.token_env:
|
||||
continue
|
||||
existing = out.get(r.token_env)
|
||||
if existing is not None and existing != r.token_ref:
|
||||
@@ -216,43 +206,35 @@ def egress_token_env_map(
|
||||
return out
|
||||
|
||||
|
||||
def _route_to_yaml_fields(r: Route) -> dict:
|
||||
"""Return the addon-visible fields for one route.
|
||||
|
||||
Single authoritative mapping between EgressRoute (host-side) and
|
||||
egress_addon_core.Route (sidecar-side). When a field is added to
|
||||
the addon's Route that must appear in the YAML, add it here and
|
||||
in egress_addon_core._parse_one together."""
|
||||
fields: dict = {"host": r.host}
|
||||
if r.auth_scheme and r.token_env:
|
||||
fields["auth_scheme"] = r.auth_scheme
|
||||
fields["token_env"] = r.token_env
|
||||
if r.path_allowlist:
|
||||
fields["path_allowlist"] = list(r.path_allowlist)
|
||||
return fields
|
||||
|
||||
|
||||
def egress_render_routes(
|
||||
routes: tuple[EgressRoute, ...],
|
||||
) -> str:
|
||||
"""Serialize the route table for the addon to read.
|
||||
|
||||
YAML content — no token values, no host env-var names. Fields are
|
||||
determined by `_route_to_yaml_fields`, which is the single point of
|
||||
truth for the EgressRoute → egress_addon_core.Route mapping."""
|
||||
YAML content — no token values, no host env-var names. The only
|
||||
thing the addon needs at runtime is the host → path_allowlist
|
||||
+ auth_scheme + in-container env-var mapping. The actual token
|
||||
values arrive via the container's environ.
|
||||
|
||||
Authenticated routes carry `auth_scheme` + `token_env`;
|
||||
unauthenticated routes omit both keys (the addon's parser
|
||||
enforces both-or-neither). Hand-rolled YAML in the style of
|
||||
`pipelock_render_yaml` so the addon's parser
|
||||
(`yaml_subset.parse_yaml_subset`) round-trips it cleanly."""
|
||||
lines: list[str] = ["routes:"]
|
||||
if not routes:
|
||||
# `routes:` with an empty list on the same line — the parser
|
||||
# needs SOMETHING here. Empty inline list is the cleanest.
|
||||
lines[0] = "routes: []"
|
||||
return "\n".join(lines) + "\n"
|
||||
for r in routes:
|
||||
f = _route_to_yaml_fields(r)
|
||||
lines.append(f' - host: "{f["host"]}"')
|
||||
if "auth_scheme" in f:
|
||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||
lines.append(f' token_env: "{f["token_env"]}"')
|
||||
if "path_allowlist" in f:
|
||||
lines.append(f' - host: "{r.host}"')
|
||||
if r.auth_scheme and r.token_env:
|
||||
lines.append(f' auth_scheme: "{r.auth_scheme}"')
|
||||
lines.append(f' token_env: "{r.token_env}"')
|
||||
if r.path_allowlist:
|
||||
lines.append(" path_allowlist:")
|
||||
for p in f["path_allowlist"]:
|
||||
for p in r.path_allowlist:
|
||||
lines.append(f' - "{p}"')
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
@@ -292,23 +274,18 @@ class Egress(ABC):
|
||||
sidecar's start/stop lifecycle is backend-specific and lives on
|
||||
concrete subclasses."""
|
||||
|
||||
def prepare(
|
||||
self,
|
||||
bottle: Bottle,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> EgressPlan:
|
||||
"""Lift `bottle.egress.routes` + `provider_routes` into resolved
|
||||
routes, render the routes file (mode 600) under `stage_dir`, and
|
||||
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> EgressPlan:
|
||||
"""Lift `bottle.egress.routes` into resolved routes,
|
||||
render the routes file (mode 600) under `stage_dir`, and
|
||||
return the plan. Pure host-side, no docker subprocess. The
|
||||
token-env map records the mapping the launch step uses to
|
||||
forward values from the host's environ into the sidecar's environ.
|
||||
forward values from the host's environ into the sidecar's
|
||||
environ.
|
||||
|
||||
Returned plan is incomplete: the launch step must fill
|
||||
`internal_network` / `egress_network` / `pipelock_proxy_url`
|
||||
via `dataclasses.replace` before passing it to `.start`."""
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||
routes = egress_routes_for_bottle(bottle)
|
||||
routes_path = stage_dir / "egress_routes.yaml"
|
||||
routes_path.write_text(egress_render_routes(routes))
|
||||
routes_path.chmod(0o600)
|
||||
@@ -320,7 +297,6 @@ class Egress(ABC):
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
|
||||
"EGRESS_HOSTNAME",
|
||||
"EGRESS_ROUTES_IN_CONTAINER",
|
||||
"Egress",
|
||||
|
||||
+51
-113
@@ -29,24 +29,22 @@ backend-specific and lives on concrete subclasses (see
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import shlex
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Mapping
|
||||
|
||||
from .log import info
|
||||
from .log import die
|
||||
from .manifest import Bottle, GitEntry
|
||||
|
||||
|
||||
# Short network alias for git-gate inside the sidecar bundle. The
|
||||
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
|
||||
GIT_GATE_HOSTNAME = "git-gate"
|
||||
# Bound half-open git client sessions. If an agent/tool runner is
|
||||
# interrupted during push, git daemon should reap the receive-pack
|
||||
# child instead of keeping the gate wedged indefinitely.
|
||||
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
|
||||
|
||||
|
||||
def _empty_str_map() -> dict[str, str]:
|
||||
return {}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -62,7 +60,10 @@ class GitGateUpstream:
|
||||
KnownHostKey string from the manifest; the gate's start step
|
||||
materialises it into a known_hosts file if non-empty.
|
||||
|
||||
the gate credential paths inside the running sidecar."""
|
||||
`extra_hosts` is a `{hostname: ip}` map the backend injects into
|
||||
the gate container's `/etc/hosts` via `--add-host` so the gate
|
||||
can resolve upstream hostnames that aren't reachable via the
|
||||
container's default DNS (e.g. Tailscale-only hosts)."""
|
||||
|
||||
name: str
|
||||
upstream_url: str
|
||||
@@ -71,6 +72,7 @@ class GitGateUpstream:
|
||||
identity_file: str
|
||||
known_host_key: str
|
||||
known_hosts_file: Path = Path()
|
||||
extra_hosts: Mapping[str, str] = field(default_factory=_empty_str_map)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -107,19 +109,46 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
|
||||
upstream_port=e.UpstreamPort,
|
||||
identity_file=e.IdentityFile,
|
||||
known_host_key=e.KnownHostKey,
|
||||
extra_hosts=dict(e.ExtraHosts),
|
||||
)
|
||||
for e in bottle.git
|
||||
)
|
||||
|
||||
|
||||
def git_gate_aggregate_extra_hosts(
|
||||
upstreams: tuple[GitGateUpstream, ...],
|
||||
) -> dict[str, str]:
|
||||
"""Merge every upstream's `extra_hosts` into a single
|
||||
`{hostname: ip}` map for `--add-host` on the gate container. Two
|
||||
entries naming the same hostname with different IPs is a manifest
|
||||
bug — the gate has one /etc/hosts — so die loudly with the
|
||||
conflicting names rather than silently picking one."""
|
||||
merged: dict[str, str] = {}
|
||||
source: dict[str, str] = {}
|
||||
for u in upstreams:
|
||||
for host, ip in u.extra_hosts.items():
|
||||
existing = merged.get(host)
|
||||
if existing is None:
|
||||
merged[host] = ip
|
||||
source[host] = u.name
|
||||
elif existing != ip:
|
||||
die(
|
||||
f"git-gate ExtraHosts conflict: '{host}' maps to "
|
||||
f"'{existing}' in upstream '{source[host]}' and to "
|
||||
f"'{ip}' in upstream '{u.name}'. The gate has one "
|
||||
f"/etc/hosts; pick one IP."
|
||||
)
|
||||
return merged
|
||||
|
||||
|
||||
def git_gate_render_gitconfig(
|
||||
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||
entries: tuple[GitEntry, ...], gate_host: str
|
||||
) -> str:
|
||||
"""Render the agent's ~/.gitconfig content for git-gate
|
||||
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
|
||||
exposed for tests + reuse across backends.
|
||||
|
||||
`gate_host` is the part of the URL between `<scheme>://` and the
|
||||
`gate_host` is the part of the URL between `git://` and the
|
||||
repo path — backends differ here:
|
||||
- docker: `git-gate` (the short network alias)
|
||||
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
|
||||
@@ -136,7 +165,7 @@ def git_gate_render_gitconfig(
|
||||
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
|
||||
]
|
||||
for entry in entries:
|
||||
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
|
||||
out.append(f'[url "git://{gate_host}/{entry.Name}.git"]\n')
|
||||
out.append(f"\tinsteadOf = {entry.Upstream}\n")
|
||||
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
|
||||
port = (
|
||||
@@ -204,20 +233,20 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
|
||||
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
|
||||
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
|
||||
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
|
||||
" git -C \"$repo\" config http.receivepack true",
|
||||
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
|
||||
"}",
|
||||
"",
|
||||
"mkdir -p /git",
|
||||
]
|
||||
for u in upstreams:
|
||||
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
|
||||
# Single-quote args so URL/path content (containing : and /)
|
||||
# passes through ash unmangled. Names came through the manifest
|
||||
# validator so they don't contain a single quote.
|
||||
lines.append(f"init_repo '{u.name}' '{u.upstream_url}'")
|
||||
lines.extend([
|
||||
"",
|
||||
"exec git daemon \\",
|
||||
" --reuseaddr \\",
|
||||
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
|
||||
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
|
||||
" --base-path=/git \\",
|
||||
" --export-all \\",
|
||||
" --enable=receive-pack \\",
|
||||
@@ -251,14 +280,7 @@ while IFS=' ' read -r old new ref; do
|
||||
[ -z "$ref" ] && continue
|
||||
[ "$new" = "$zero" ] && continue
|
||||
if [ "$old" = "$zero" ]; then
|
||||
# New ref: scan only the commits this push introduces — those
|
||||
# reachable from $new but not from any ref the gate already has.
|
||||
# Everything already on the gate arrived via upstream mirror-fetch
|
||||
# or a previously gitleaks-scanned push, so it's already-upstream
|
||||
# or already-scanned; re-scanning it (the old `$new` full-ancestry
|
||||
# range) only resurfaces historical findings and blocks every new
|
||||
# branch. See PRD 0028 / issue #106.
|
||||
log_opts="$new --not --all"
|
||||
log_opts="$new"
|
||||
else
|
||||
log_opts="$old..$new"
|
||||
fi
|
||||
@@ -278,7 +300,7 @@ if [ ! -f "$hostsfile" ]; then
|
||||
echo "git-gate: add KnownHostKey to the bottle.git entry and restart the bottle" >&2
|
||||
exit 1
|
||||
fi
|
||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
|
||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes"
|
||||
|
||||
while IFS=' ' read -r old new ref; do
|
||||
[ -z "$ref" ] && continue
|
||||
@@ -333,7 +355,7 @@ if [ -z "$keyfile" ] || [ ! -f "$hostsfile" ]; then
|
||||
echo "git-gate: missing credentials for $repo_dir; refusing fetch" >&2
|
||||
exit 1
|
||||
fi
|
||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
|
||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes"
|
||||
|
||||
echo "git-gate: refreshing $repo_dir from upstream" >&2
|
||||
if ! GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" fetch origin --prune >&2; then
|
||||
@@ -360,80 +382,6 @@ exit 0
|
||||
"""
|
||||
|
||||
|
||||
def _provision_dynamic_key(
|
||||
entry: GitEntry,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
) -> str:
|
||||
"""Generate a fresh ed25519 keypair, register the public half with
|
||||
the forge, and persist the private key + key ID under `stage_dir`.
|
||||
|
||||
Returns the host-side path to the private key file so the caller
|
||||
can inject it into the GitGateUpstream as `identity_file`."""
|
||||
from .deploy_key_provisioner import get_provisioner
|
||||
pk = entry.ProvisionedKey
|
||||
assert pk is not None
|
||||
token = os.environ.get(pk.token_env)
|
||||
if token is None:
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||
f" = {pk.token_env!r}: env var is not set"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||
|
||||
owner_repo = entry.UpstreamPath
|
||||
if owner_repo.endswith(".git"):
|
||||
owner_repo = owner_repo[:-4]
|
||||
title = f"bot-bottle:{slug}:{entry.Name}"
|
||||
|
||||
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
|
||||
key_id, private_key_bytes = provisioner.create(owner_repo, title)
|
||||
|
||||
key_file = stage_dir / f"{entry.Name}-key"
|
||||
key_file.write_bytes(private_key_bytes)
|
||||
key_file.chmod(0o600)
|
||||
|
||||
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
||||
id_file.write_text(key_id)
|
||||
id_file.chmod(0o600)
|
||||
|
||||
info(f"provisioned deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||
return str(key_file)
|
||||
|
||||
|
||||
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
||||
"""Revoke all deploy keys provisioned for `bottle` during prepare.
|
||||
|
||||
Called at teardown after containers stop. Raises if any revocation
|
||||
fails — a stranded key is a security concern that the operator must
|
||||
address manually."""
|
||||
from .deploy_key_provisioner import get_provisioner
|
||||
for entry in bottle.git:
|
||||
if entry.ProvisionedKey is None:
|
||||
continue
|
||||
pk = entry.ProvisionedKey
|
||||
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
||||
if not id_file.exists():
|
||||
continue
|
||||
key_id = id_file.read_text().strip()
|
||||
token = os.environ.get(pk.token_env)
|
||||
if token is None:
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||
f" = {pk.token_env!r}: env var is not set;"
|
||||
f" cannot revoke deploy key {key_id}"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||
owner_repo = entry.UpstreamPath
|
||||
if owner_repo.endswith(".git"):
|
||||
owner_repo = owner_repo[:-4]
|
||||
info(f"revoking deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||
provisioner.delete(owner_repo, key_id)
|
||||
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||
|
||||
|
||||
class GitGate(ABC):
|
||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||
(upstream lift + entrypoint/hook render); the sidecar's
|
||||
@@ -445,21 +393,10 @@ class GitGate(ABC):
|
||||
entrypoint, pre-receive hook, and access-hook scripts (mode
|
||||
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
||||
|
||||
For `provisioned_key` entries, also generates and registers
|
||||
a fresh deploy key via the forge API and writes the private key
|
||||
+ key ID to `stage_dir`.
|
||||
|
||||
Returned plan is incomplete: the launch step must fill
|
||||
`internal_network` / `egress_network` via `dataclasses.replace`
|
||||
before passing the plan to `.start`."""
|
||||
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
|
||||
for i, entry in enumerate(bottle.git):
|
||||
if entry.ProvisionedKey is not None:
|
||||
key_file = _provision_dynamic_key(entry, slug, stage_dir)
|
||||
upstreams_list[i] = dataclasses.replace(
|
||||
upstreams_list[i], identity_file=key_file
|
||||
)
|
||||
upstreams = tuple(upstreams_list)
|
||||
upstreams = git_gate_upstreams_for_bottle(bottle)
|
||||
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
||||
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
||||
entrypoint.chmod(0o600)
|
||||
@@ -492,6 +429,7 @@ class GitGate(ABC):
|
||||
identity_file=u.identity_file,
|
||||
known_host_key=u.known_host_key,
|
||||
known_hosts_file=known_hosts_file,
|
||||
extra_hosts=dict(u.extra_hosts),
|
||||
)
|
||||
)
|
||||
return GitGatePlan(
|
||||
|
||||
@@ -1,175 +0,0 @@
|
||||
"""Tiny smart-HTTP wrapper for git-gate repos.
|
||||
|
||||
Used by the smolmachines backend where `git://` push traffic over the
|
||||
host-published Docker port can hang before receive-pack reaches hooks.
|
||||
The wrapper serves the same `/git/*.git` bare repos through
|
||||
`git http-backend`, so pre-receive and upstream forwarding remain the
|
||||
git-gate enforcement point.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
|
||||
DEFAULT_PORT = 9420
|
||||
|
||||
# Body-size cap matching supervise_server.py's 1 MiB limit.
|
||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||
|
||||
|
||||
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
server_version = "bot-bottle-git-http/1"
|
||||
|
||||
def do_GET(self) -> None:
|
||||
self._run_backend()
|
||||
|
||||
def do_POST(self) -> None:
|
||||
self._run_backend()
|
||||
|
||||
def _run_backend(self) -> None:
|
||||
parsed = urlsplit(self.path)
|
||||
if self._is_upload_pack(parsed.path, parsed.query):
|
||||
repo_dir = self._repo_dir(parsed.path)
|
||||
if repo_dir is None:
|
||||
self.send_error(404)
|
||||
return
|
||||
hook_path = os.environ.get(
|
||||
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
|
||||
)
|
||||
peer = self.client_address[0]
|
||||
hook = subprocess.run(
|
||||
[hook_path, "upload-pack", str(repo_dir), peer, peer],
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
if hook.returncode != 0:
|
||||
detail = (hook.stderr or hook.stdout).decode(
|
||||
"utf-8", errors="replace",
|
||||
).rstrip()
|
||||
if detail:
|
||||
for line in detail.splitlines():
|
||||
self.log_message("access-hook denied %s: %s",
|
||||
parsed.path, line)
|
||||
else:
|
||||
self.log_message(
|
||||
"access-hook denied %s: exit=%d (no output)",
|
||||
parsed.path, hook.returncode,
|
||||
)
|
||||
self.send_response(403)
|
||||
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||
self.end_headers()
|
||||
self.wfile.write(hook.stderr or hook.stdout)
|
||||
return
|
||||
env = os.environ.copy()
|
||||
env.update({
|
||||
"GIT_PROJECT_ROOT": os.environ.get("GIT_PROJECT_ROOT", "/git"),
|
||||
"GIT_HTTP_EXPORT_ALL": "1",
|
||||
"REQUEST_METHOD": self.command,
|
||||
"PATH_INFO": parsed.path,
|
||||
"QUERY_STRING": parsed.query,
|
||||
"CONTENT_TYPE": self.headers.get("content-type", ""),
|
||||
"CONTENT_LENGTH": self.headers.get("content-length", "0"),
|
||||
"REMOTE_ADDR": self.client_address[0],
|
||||
"REMOTE_PORT": str(self.client_address[1]),
|
||||
"REMOTE_USER": "",
|
||||
"SERVER_NAME": self.server.server_name,
|
||||
"SERVER_PORT": str(self.server.server_port),
|
||||
"SERVER_PROTOCOL": self.request_version,
|
||||
})
|
||||
for header, variable in (
|
||||
("accept", "HTTP_ACCEPT"),
|
||||
("content-encoding", "HTTP_CONTENT_ENCODING"),
|
||||
("git-protocol", "HTTP_GIT_PROTOCOL"),
|
||||
("user-agent", "HTTP_USER_AGENT"),
|
||||
):
|
||||
value = self.headers.get(header)
|
||||
if value:
|
||||
env[variable] = value
|
||||
raw_length = self.headers.get("content-length", "0") or "0"
|
||||
try:
|
||||
length = int(raw_length)
|
||||
except ValueError:
|
||||
self.send_error(400, "Bad Content-Length")
|
||||
return
|
||||
if length < 0:
|
||||
self.send_error(400, "Negative Content-Length")
|
||||
return
|
||||
if length > MAX_BODY_BYTES:
|
||||
self.send_error(413, "Request body too large")
|
||||
return
|
||||
body = self.rfile.read(length) if length else b""
|
||||
proc = subprocess.run(
|
||||
["git", "http-backend"],
|
||||
input=body,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
self._write_cgi_response(proc.stdout)
|
||||
|
||||
def _repo_dir(self, path: str) -> Path | None:
|
||||
root = Path(os.environ.get("GIT_PROJECT_ROOT", "/git")).resolve()
|
||||
relative = path.lstrip("/").split(".git", 1)[0] + ".git"
|
||||
candidate = (root / relative).resolve()
|
||||
if root not in (candidate, *candidate.parents):
|
||||
return None
|
||||
if not candidate.is_dir():
|
||||
return None
|
||||
return candidate
|
||||
|
||||
@staticmethod
|
||||
def _is_upload_pack(path: str, query: str) -> bool:
|
||||
if path.endswith("/git-upload-pack"):
|
||||
return True
|
||||
if path.endswith("/info/refs"):
|
||||
return any(
|
||||
pair == "service=git-upload-pack"
|
||||
for pair in query.split("&")
|
||||
)
|
||||
return False
|
||||
|
||||
def _write_cgi_response(self, raw: bytes) -> None:
|
||||
head, sep, body = raw.partition(b"\r\n\r\n")
|
||||
line_sep = b"\r\n"
|
||||
if not sep:
|
||||
head, sep, body = raw.partition(b"\n\n")
|
||||
line_sep = b"\n"
|
||||
status = 200
|
||||
headers: list[tuple[str, str]] = []
|
||||
for line in head.split(line_sep):
|
||||
if not line:
|
||||
continue
|
||||
key, _, value = line.decode("latin1").partition(":")
|
||||
value = value.strip()
|
||||
if key.lower() == "status":
|
||||
status = int(value.split()[0])
|
||||
else:
|
||||
headers.append((key, value))
|
||||
self.send_response(status)
|
||||
for key, value in headers:
|
||||
self.send_header(key, value)
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, fmt: str, *args: object) -> None:
|
||||
sys.stdout.write(fmt % args + "\n")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
|
||||
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
|
||||
sys.stdout.write(f"git-http listening on 0.0.0.0:{port}\n")
|
||||
sys.stdout.flush()
|
||||
server.serve_forever()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
+3
-15
@@ -14,23 +14,11 @@ def warn(msg: str) -> None:
|
||||
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
def error(msg: str) -> None:
|
||||
print(f"bot-bottle: error: {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
class Die(SystemExit):
|
||||
"""Raised by die() so callers (and tests) can distinguish a deliberate
|
||||
fatal exit from an unrelated SystemExit.
|
||||
|
||||
Carries the human-facing message so a caller that suppressed stderr
|
||||
— e.g. the curses dashboard, whose alternate screen is wiped when the
|
||||
terminal is restored — can re-surface the reason after the fact."""
|
||||
|
||||
def __init__(self, code: int = 1, message: str = "") -> None:
|
||||
super().__init__(code)
|
||||
self.message = message
|
||||
fatal exit from an unrelated SystemExit."""
|
||||
|
||||
|
||||
def die(msg: str) -> NoReturn:
|
||||
error(msg)
|
||||
raise Die(1, msg)
|
||||
print(f"bot-bottle: error: {msg}", file=sys.stderr)
|
||||
raise Die(1)
|
||||
|
||||
+1051
-126
File diff suppressed because it is too large
Load Diff
@@ -1,166 +0,0 @@
|
||||
"""Agent configuration manifest dataclasses."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import cast
|
||||
|
||||
from .agent_provider import PROVIDER_TEMPLATES
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
from .manifest_git import GitUser
|
||||
from .manifest_schema import AGENT_MODEL_KEYS
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvider:
|
||||
"""Provider/template for the agent process inside a bottle.
|
||||
|
||||
`template` selects a built-in launch/runtime contract. `dockerfile`
|
||||
optionally points at a custom agent-image Dockerfile while leaving
|
||||
bot-bottle's sidecar infrastructure intact.
|
||||
|
||||
`auth_token` names the host env var that holds the provider's OAuth
|
||||
token (Claude only). The provisioner injects a provider-owned egress
|
||||
route for api.anthropic.com that re-injects this token as the Bearer
|
||||
header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent
|
||||
so the Claude Code CLI starts.
|
||||
|
||||
`forward_host_credentials` forwards the host Codex auth token into
|
||||
the egress sidecar (Codex only).
|
||||
"""
|
||||
|
||||
template: str = "claude"
|
||||
dockerfile: str = ""
|
||||
auth_token: str = ""
|
||||
forward_host_credentials: bool = False
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
|
||||
for k in d:
|
||||
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
|
||||
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
|
||||
)
|
||||
template = d.get("template", "claude")
|
||||
if not isinstance(template, str) or not template:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.template must be a "
|
||||
f"non-empty string"
|
||||
)
|
||||
if template not in PROVIDER_TEMPLATES:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.template {template!r} "
|
||||
f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
|
||||
)
|
||||
dockerfile = d.get("dockerfile", "")
|
||||
if not isinstance(dockerfile, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.dockerfile must be a "
|
||||
f"string (was {type(dockerfile).__name__})"
|
||||
)
|
||||
auth_token = d.get("auth_token", "")
|
||||
if not isinstance(auth_token, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
|
||||
f"string (was {type(auth_token).__name__})"
|
||||
)
|
||||
if auth_token and template != "claude":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
||||
f"supported for template 'claude'"
|
||||
)
|
||||
forward_host_credentials = d.get("forward_host_credentials", False)
|
||||
if not isinstance(forward_host_credentials, bool):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||
f"must be a boolean (was {type(forward_host_credentials).__name__})"
|
||||
)
|
||||
if forward_host_credentials and template != "codex":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||
"is currently only supported for template 'codex'"
|
||||
)
|
||||
return cls(
|
||||
template=template,
|
||||
dockerfile=dockerfile,
|
||||
auth_token=auth_token,
|
||||
forward_host_credentials=forward_host_credentials,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Agent:
|
||||
bottle: str
|
||||
skills: tuple[str, ...] = ()
|
||||
prompt: str = ""
|
||||
# Per-agent git identity (issue #94). Overlays the referenced
|
||||
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
||||
# `user` is allowed at the agent level; `repos` stays bottle-only
|
||||
# because it carries credentials and host trust.
|
||||
git_user: GitUser = GitUser()
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
|
||||
d = as_json_object(raw, f"agent '{name}'")
|
||||
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
||||
if unknown:
|
||||
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
|
||||
raise ManifestError(
|
||||
f"agent '{name}' has unknown key(s) {sorted(unknown)}; "
|
||||
f"allowed keys are {allowed}."
|
||||
)
|
||||
|
||||
bottle = d.get("bottle")
|
||||
if not isinstance(bottle, str) or not bottle:
|
||||
raise ManifestError(f"agent '{name}' must declare a 'bottle' field naming a defined bottle")
|
||||
if bottle not in bottle_names:
|
||||
available = ", ".join(sorted(bottle_names)) or "(none defined)"
|
||||
raise ManifestError(
|
||||
f"agent '{name}' references bottle '{bottle}', which is not defined. "
|
||||
f"Available: {available}"
|
||||
)
|
||||
|
||||
skills: tuple[str, ...] = ()
|
||||
skills_raw = d.get("skills")
|
||||
if skills_raw is not None:
|
||||
if not isinstance(skills_raw, list):
|
||||
raise ManifestError(f"agent '{name}' skills must be an array (was {type(skills_raw).__name__})")
|
||||
collected: list[str] = []
|
||||
skills_list = cast(list[object], skills_raw)
|
||||
for i, skill in enumerate(skills_list):
|
||||
if not isinstance(skill, str):
|
||||
raise ManifestError(
|
||||
f"agent '{name}' skills[{i}] must be a string "
|
||||
f"(was {type(skill).__name__})"
|
||||
)
|
||||
collected.append(skill)
|
||||
skills = tuple(collected)
|
||||
|
||||
prompt_raw = d.get("prompt")
|
||||
if prompt_raw is None:
|
||||
prompt = ""
|
||||
elif isinstance(prompt_raw, str):
|
||||
prompt = prompt_raw
|
||||
else:
|
||||
raise ManifestError(f"agent '{name}' prompt must be a string (was {type(prompt_raw).__name__})")
|
||||
|
||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||
git_user = GitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||
for k in gd.keys():
|
||||
if k != "user":
|
||||
raise ManifestError(
|
||||
f"agent '{name}' git-gate.{k} is not allowed at the "
|
||||
f"agent level; only git-gate.user (name/email) may be "
|
||||
f"set on an agent. git-gate.repos is bottle-only "
|
||||
f"(it carries credentials and host trust)."
|
||||
)
|
||||
if "user" in gd:
|
||||
git_user = GitUser.from_dict(name, gd["user"])
|
||||
|
||||
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
||||
@@ -1,286 +0,0 @@
|
||||
"""Egress routing manifest dataclasses and helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
from dataclasses import dataclass, field
|
||||
from typing import cast
|
||||
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
|
||||
|
||||
# Auth schemes for the egress route's optional `auth` block.
|
||||
# Same values cred-proxy accepts today; `token` sidesteps the Gitea
|
||||
# token-not-Bearer quirk (go-gitea/gitea#16734).
|
||||
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
|
||||
|
||||
|
||||
def validate_egress_routes(
|
||||
bottle_name: str,
|
||||
routes: tuple[EgressRoute, ...],
|
||||
) -> None:
|
||||
"""Cross-validation for `bottle.egress.routes`: hosts must be unique.
|
||||
|
||||
The proxy matches by exact-host (v1); duplicate hosts leave the
|
||||
route choice ambiguous so we reject them up front.
|
||||
|
||||
No cross-validation against `bottle.git-gate.repos` is performed.
|
||||
git-gate (SSH push/fetch) and egress (HTTPS) broker different
|
||||
protocols; declaring both for the same host is a legitimate dev
|
||||
setup."""
|
||||
seen_hosts: dict[str, None] = {}
|
||||
for r in routes:
|
||||
key = r.Host.lower()
|
||||
if key in seen_hosts:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' egress.routes has duplicate host "
|
||||
f"{r.Host!r}; each host must be unique on the proxy."
|
||||
)
|
||||
seen_hosts[key] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PipelockRoutePolicy:
|
||||
"""Per-route pipelock policy overrides.
|
||||
|
||||
`TlsPassthrough` adds the route host to pipelock's
|
||||
`tls_interception.passthrough_domains`, so pipelock still enforces
|
||||
the hostname allowlist but does not MITM/decrypt request bodies or
|
||||
headers for that host.
|
||||
|
||||
`SsrfIpAllowlist` adds explicit IPs/CIDRs to pipelock's SSRF
|
||||
allowlist for private/internal destinations behind this route.
|
||||
"""
|
||||
|
||||
TlsPassthrough: bool = False
|
||||
SsrfIpAllowlist: tuple[str, ...] = ()
|
||||
|
||||
@classmethod
|
||||
def from_dict(
|
||||
cls, bottle_name: str, idx: int, raw: object,
|
||||
) -> "PipelockRoutePolicy":
|
||||
label = f"bottle '{bottle_name}' egress.routes[{idx}] pipelock"
|
||||
d = as_json_object(raw, label)
|
||||
for k in d:
|
||||
if k not in ("tls_passthrough", "ssrf_ip_allowlist"):
|
||||
raise ManifestError(
|
||||
f"{label} has unknown key {k!r}; "
|
||||
f"only 'tls_passthrough' and 'ssrf_ip_allowlist' "
|
||||
f"are accepted"
|
||||
)
|
||||
tls_passthrough_raw = d.get("tls_passthrough", False)
|
||||
if not isinstance(tls_passthrough_raw, bool):
|
||||
raise ManifestError(
|
||||
f"{label}.tls_passthrough must be a boolean "
|
||||
f"(was {type(tls_passthrough_raw).__name__})"
|
||||
)
|
||||
ssrf_raw = d.get("ssrf_ip_allowlist", [])
|
||||
if not isinstance(ssrf_raw, list):
|
||||
raise ManifestError(
|
||||
f"{label}.ssrf_ip_allowlist must be an array "
|
||||
f"(was {type(ssrf_raw).__name__})"
|
||||
)
|
||||
ssrf_ip_allowlist: list[str] = []
|
||||
for j, item in enumerate(ssrf_raw):
|
||||
if not isinstance(item, str) or not item:
|
||||
raise ManifestError(
|
||||
f"{label}.ssrf_ip_allowlist[{j}] must be a non-empty "
|
||||
f"string (was {type(item).__name__})"
|
||||
)
|
||||
try:
|
||||
ipaddress.ip_network(item, strict=False)
|
||||
except ValueError as e:
|
||||
raise ManifestError(
|
||||
f"{label}.ssrf_ip_allowlist[{j}] must be an IP address "
|
||||
f"or CIDR (was {item!r}): {e}"
|
||||
)
|
||||
ssrf_ip_allowlist.append(item)
|
||||
return cls(
|
||||
TlsPassthrough=tls_passthrough_raw,
|
||||
SsrfIpAllowlist=tuple(ssrf_ip_allowlist),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressRoute:
|
||||
"""One route on the per-bottle egress sidecar (PRD 0017).
|
||||
|
||||
`Host` matches the request's hostname (case-insensitive). The
|
||||
optional `PathAllowlist` constrains the URL path to a set of
|
||||
prefixes; empty tuple means no path-level filtering. The optional
|
||||
`AuthScheme` / `TokenRef` pair drives credential injection:
|
||||
when set, the proxy strips any inbound Authorization and injects
|
||||
`<AuthScheme> <value-of-host-env-named-by-TokenRef>`. When the
|
||||
manifest's `auth` block is omitted both fields are empty strings —
|
||||
no Authorization is written, no token forwarded.
|
||||
|
||||
`Role` is reserved for future use; all role strings are currently
|
||||
rejected by the validator.
|
||||
|
||||
Validation rules (enforced in `from_dict`):
|
||||
- `host` required, non-empty.
|
||||
- `path_allowlist` optional, list of absolute path prefixes.
|
||||
- `auth` optional. If present, MUST carry both `scheme` and
|
||||
`token_ref` as non-empty strings; an empty `auth: {}` is an
|
||||
error rather than a synonym for "no auth" (omit `auth` for
|
||||
that case).
|
||||
- `role` optional, reserved — any non-empty value is rejected.
|
||||
"""
|
||||
|
||||
Host: str
|
||||
PathAllowlist: tuple[str, ...] = ()
|
||||
AuthScheme: str = ""
|
||||
TokenRef: str = ""
|
||||
Role: tuple[str, ...] = ()
|
||||
Pipelock: PipelockRoutePolicy = field(default_factory=PipelockRoutePolicy)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
||||
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
|
||||
d = as_json_object(raw, label)
|
||||
host = d.get("host")
|
||||
if not isinstance(host, str) or not host:
|
||||
raise ManifestError(f"{label} missing required string field 'host'")
|
||||
|
||||
path_allow_raw = d.get("path_allowlist")
|
||||
prefixes: tuple[str, ...] = ()
|
||||
if path_allow_raw is not None:
|
||||
if not isinstance(path_allow_raw, list):
|
||||
raise ManifestError(
|
||||
f"{label} path_allowlist must be an array "
|
||||
f"(was {type(path_allow_raw).__name__})"
|
||||
)
|
||||
path_list = cast(list[object], path_allow_raw)
|
||||
collected: list[str] = []
|
||||
for j, p in enumerate(path_list):
|
||||
if not isinstance(p, str):
|
||||
raise ManifestError(
|
||||
f"{label} path_allowlist[{j}] must be a string "
|
||||
f"(was {type(p).__name__})"
|
||||
)
|
||||
if not p.startswith("/"):
|
||||
raise ManifestError(
|
||||
f"{label} path_allowlist[{j}] {p!r} must be an "
|
||||
f"absolute path prefix starting with '/'"
|
||||
)
|
||||
collected.append(p)
|
||||
prefixes = tuple(collected)
|
||||
|
||||
auth_scheme = ""
|
||||
token_ref = ""
|
||||
if "auth" in d:
|
||||
auth_raw = d.get("auth")
|
||||
auth_d = as_json_object(auth_raw, f"{label} auth")
|
||||
if not auth_d:
|
||||
raise ManifestError(
|
||||
f"{label} auth is empty ({{}}); omit the 'auth' key "
|
||||
f"entirely if this route is unauthenticated. Otherwise "
|
||||
f"both 'scheme' and 'token_ref' are required."
|
||||
)
|
||||
auth_scheme_raw = auth_d.get("scheme")
|
||||
if not isinstance(auth_scheme_raw, str) or not auth_scheme_raw:
|
||||
raise ManifestError(
|
||||
f"{label} auth.scheme is required when 'auth' is set "
|
||||
f"(non-empty string)"
|
||||
)
|
||||
if auth_scheme_raw not in EGRESS_AUTH_SCHEMES:
|
||||
raise ManifestError(
|
||||
f"{label} auth.scheme {auth_scheme_raw!r} is not one of "
|
||||
f"{', '.join(EGRESS_AUTH_SCHEMES)}"
|
||||
)
|
||||
token_ref_raw = auth_d.get("token_ref")
|
||||
if not isinstance(token_ref_raw, str) or not token_ref_raw:
|
||||
raise ManifestError(
|
||||
f"{label} auth.token_ref is required when 'auth' is set "
|
||||
f"(name of the host env var holding the token value)"
|
||||
)
|
||||
for k in auth_d:
|
||||
if k not in ("scheme", "token_ref"):
|
||||
raise ManifestError(
|
||||
f"{label} auth has unknown key {k!r}; "
|
||||
f"only 'scheme' and 'token_ref' are accepted"
|
||||
)
|
||||
auth_scheme = auth_scheme_raw
|
||||
token_ref = token_ref_raw
|
||||
|
||||
role_raw = d.get("role")
|
||||
roles: tuple[str, ...] = ()
|
||||
if role_raw is None:
|
||||
roles = ()
|
||||
elif isinstance(role_raw, str):
|
||||
roles = (role_raw,)
|
||||
elif isinstance(role_raw, list):
|
||||
role_list = cast(list[object], role_raw)
|
||||
collected_roles: list[str] = []
|
||||
for r in role_list:
|
||||
if not isinstance(r, str):
|
||||
raise ManifestError(f"{label} role items must be strings (got {type(r).__name__})")
|
||||
collected_roles.append(r)
|
||||
roles = tuple(collected_roles)
|
||||
else:
|
||||
raise ManifestError(
|
||||
f"{label} role must be a string or a list of strings "
|
||||
f"(was {type(role_raw).__name__})"
|
||||
)
|
||||
if roles:
|
||||
raise ManifestError(
|
||||
f"{label} role {roles[0]!r} is not accepted; "
|
||||
f"the 'role' field is reserved for future use"
|
||||
)
|
||||
|
||||
pipelock = (
|
||||
PipelockRoutePolicy.from_dict(bottle_name, idx, d["pipelock"])
|
||||
if "pipelock" in d
|
||||
else PipelockRoutePolicy()
|
||||
)
|
||||
|
||||
for k in d:
|
||||
if k not in ("host", "path_allowlist", "auth", "role", "pipelock"):
|
||||
raise ManifestError(
|
||||
f"{label} has unknown key {k!r}; accepted keys are "
|
||||
f"'host', 'path_allowlist', 'auth', 'role', 'pipelock'"
|
||||
)
|
||||
|
||||
return cls(
|
||||
Host=host,
|
||||
PathAllowlist=prefixes,
|
||||
AuthScheme=auth_scheme,
|
||||
TokenRef=token_ref,
|
||||
Role=roles,
|
||||
Pipelock=pipelock,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressConfig:
|
||||
"""Per-bottle egress configuration. Today this is just the
|
||||
route table; the nesting under `egress:` leaves room for
|
||||
per-bottle proxy settings (port override, log level, etc.) in
|
||||
follow-ups."""
|
||||
|
||||
routes: tuple[EgressRoute, ...] = ()
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
|
||||
routes_raw = d.get("routes")
|
||||
routes: tuple[EgressRoute, ...] = ()
|
||||
if routes_raw is not None:
|
||||
if not isinstance(routes_raw, list):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' egress.routes must be an array "
|
||||
f"(was {type(routes_raw).__name__})"
|
||||
)
|
||||
routes_list = cast(list[object], routes_raw)
|
||||
routes = tuple(
|
||||
EgressRoute.from_dict(bottle_name, i, entry)
|
||||
for i, entry in enumerate(routes_list)
|
||||
)
|
||||
validate_egress_routes(bottle_name, routes)
|
||||
for k in d:
|
||||
if k != "routes":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' egress has unknown key {k!r}; "
|
||||
f"only 'routes' is accepted"
|
||||
)
|
||||
return cls(routes=routes)
|
||||
@@ -1,142 +0,0 @@
|
||||
"""Internal bottle `extends:` resolution for manifests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Bottle, GitEntry
|
||||
|
||||
|
||||
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
|
||||
"""Apply `extends:` chains and return resolved Bottle objects."""
|
||||
cache: dict[str, Bottle] = {}
|
||||
for name in raws:
|
||||
if name not in cache:
|
||||
_resolve_one_bottle(name, raws, cache, ())
|
||||
return cache
|
||||
|
||||
|
||||
def _resolve_one_bottle(
|
||||
name: str,
|
||||
raws: dict[str, dict[str, object]],
|
||||
cache: dict[str, Bottle],
|
||||
seen: tuple[str, ...],
|
||||
) -> Bottle:
|
||||
from .manifest import Bottle, ManifestError
|
||||
|
||||
if name in cache:
|
||||
return cache[name]
|
||||
if name in seen:
|
||||
chain = " -> ".join(seen + (name,))
|
||||
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
|
||||
raw = raws[name]
|
||||
parent_name_raw = raw.get("extends")
|
||||
# Strip `extends:` before passing to Bottle.from_dict so it
|
||||
# is not accidentally treated as a real Bottle field by future
|
||||
# schema additions. It is only meaningful here.
|
||||
child_raw = {k: v for k, v in raw.items() if k != "extends"}
|
||||
|
||||
if parent_name_raw is None:
|
||||
bottle = Bottle.from_dict(name, child_raw)
|
||||
cache[name] = bottle
|
||||
return bottle
|
||||
|
||||
if not isinstance(parent_name_raw, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' extends must be a string "
|
||||
f"(was {type(parent_name_raw).__name__})"
|
||||
)
|
||||
parent_name: str = parent_name_raw
|
||||
if parent_name == name:
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' extends itself; remove the "
|
||||
f"self-reference"
|
||||
)
|
||||
if parent_name not in raws:
|
||||
avail = ", ".join(sorted(raws.keys())) or "(none)"
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' extends '{parent_name}' which is not "
|
||||
f"defined. Available bottles: {avail}"
|
||||
)
|
||||
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
|
||||
bottle = _merge_bottles(parent, child_raw, name)
|
||||
cache[name] = bottle
|
||||
return bottle
|
||||
|
||||
|
||||
def _merge_bottles(
|
||||
parent: Bottle,
|
||||
child_raw: dict[str, object],
|
||||
name: str,
|
||||
) -> Bottle:
|
||||
"""Apply PRD 0025 merge rules."""
|
||||
from .manifest import Bottle, GitUser
|
||||
from .manifest_egress import validate_egress_routes
|
||||
|
||||
# Parse the child's declared fields into a Bottle (with the
|
||||
# usual defaults for anything missing). Validation runs the same
|
||||
# way it would for a leaf bottle: typos / wrong types die here.
|
||||
child = Bottle.from_dict(name, child_raw)
|
||||
|
||||
# env: dict merge, child wins on collision.
|
||||
merged_env = {**parent.env, **child.env}
|
||||
|
||||
# git-gate.user: per-field overlay. Each non-empty field on child
|
||||
# wins; empties fall through to parent. The default GitUser()
|
||||
# is two empty strings, so a child that omits git-gate.user
|
||||
# inherits the parent's user verbatim.
|
||||
merged_git_user = GitUser(
|
||||
name=child.git_user.name or parent.git_user.name,
|
||||
email=child.git_user.email or parent.git_user.email,
|
||||
)
|
||||
|
||||
# git-gate.repos: missing means inherit; an explicit empty object
|
||||
# clears; otherwise parent and child merge by UpstreamHost with
|
||||
# child entries replacing duplicate hosts.
|
||||
if _child_declares_git_gate_repos(child_raw):
|
||||
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
|
||||
else:
|
||||
merged_git = parent.git
|
||||
|
||||
# Presence-driven full-replace for the remaining list-valued +
|
||||
# scalar fields.
|
||||
merged_egress = child.egress if "egress" in child_raw else parent.egress
|
||||
merged_agent_provider = (
|
||||
child.agent_provider
|
||||
if "agent_provider" in child_raw
|
||||
else parent.agent_provider
|
||||
)
|
||||
merged_supervise = (
|
||||
child.supervise if "supervise" in child_raw else parent.supervise
|
||||
)
|
||||
validate_egress_routes(name, merged_egress.routes)
|
||||
|
||||
return Bottle(
|
||||
env=merged_env,
|
||||
agent_provider=merged_agent_provider,
|
||||
git=merged_git,
|
||||
git_user=merged_git_user,
|
||||
egress=merged_egress,
|
||||
supervise=merged_supervise,
|
||||
)
|
||||
|
||||
|
||||
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
git_raw = child_raw.get("git-gate")
|
||||
if git_raw is None:
|
||||
return False
|
||||
git_obj = as_json_object(git_raw, "child git-gate")
|
||||
return "repos" in git_obj
|
||||
|
||||
|
||||
def _merge_git_remotes(
|
||||
parent: tuple[GitEntry, ...],
|
||||
child: tuple[GitEntry, ...],
|
||||
) -> tuple[GitEntry, ...]:
|
||||
by_host = {entry.UpstreamHost: entry for entry in parent}
|
||||
for entry in child:
|
||||
by_host[entry.UpstreamHost] = entry
|
||||
return tuple(by_host.values())
|
||||
@@ -1,301 +0,0 @@
|
||||
"""Git-related manifest dataclasses and helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
|
||||
# Shell-safe characters for git-gate repo names. Names are embedded in
|
||||
# the generated entrypoint shell script (shlex.quote is the primary
|
||||
# defence; this regex is belt-and-suspenders and documents intent).
|
||||
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
|
||||
|
||||
|
||||
def _opt_str(value: object, label: str) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if not isinstance(value, str):
|
||||
raise ManifestError(f"{label} must be a string (was {type(value).__name__})")
|
||||
return value
|
||||
|
||||
|
||||
def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
|
||||
"""Parse `ssh://user@host[:port]/path` into (user, host, port, path).
|
||||
Dies if `url` doesn't match the ssh:// shape v1 supports. Default
|
||||
port is 22 (matches OpenSSH)."""
|
||||
if not url.startswith("ssh://"):
|
||||
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
|
||||
rest = url[len("ssh://"):]
|
||||
if "@" not in rest:
|
||||
raise ManifestError(f"{label} must include a user (e.g. ssh://git@host/path.git); was {url!r}")
|
||||
user, _, hostpart = rest.partition("@")
|
||||
if not user:
|
||||
raise ManifestError(f"{label} user is empty in {url!r}")
|
||||
if "/" not in hostpart:
|
||||
raise ManifestError(f"{label} must include a path (e.g. ssh://git@host/path.git); was {url!r}")
|
||||
hostport, _, path = hostpart.partition("/")
|
||||
if not path:
|
||||
raise ManifestError(f"{label} path is empty in {url!r}")
|
||||
if ":" in hostport:
|
||||
host, _, port = hostport.partition(":")
|
||||
if not port.isdigit():
|
||||
raise ManifestError(f"{label} port must be numeric in {url!r}")
|
||||
else:
|
||||
host = hostport
|
||||
port = "22"
|
||||
if not host:
|
||||
raise ManifestError(f"{label} host is empty in {url!r}")
|
||||
return (user, host, port, path)
|
||||
|
||||
|
||||
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
|
||||
seen: dict[str, None] = {}
|
||||
for g in git:
|
||||
if g.Name in seen:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.repos has duplicate name '{g.Name}'; "
|
||||
f"each entry maps to a distinct bare repo on the gate."
|
||||
)
|
||||
seen[g.Name] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProvisionedKeyConfig:
|
||||
"""Configuration for automatic deploy-key lifecycle management
|
||||
(PRD 0048). Used when a git-gate.repos entry opts out of a
|
||||
static identity file and instead wants a fresh SSH keypair
|
||||
generated at spin-up and revoked at teardown.
|
||||
|
||||
`provider` names the contrib sub-package to load (e.g. `gitea`).
|
||||
`token_env` is the name of a host-side env var carrying the API
|
||||
token; the value is read at provision time, never stored on the
|
||||
plan. `api_url` is the forge's HTTP API root; if empty, it is
|
||||
derived from the upstream URL's host at provision time."""
|
||||
|
||||
provider: str
|
||||
token_env: str
|
||||
api_url: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitEntry:
|
||||
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
||||
talk to. `Upstream` is the real remote URL the agent would push to
|
||||
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
|
||||
and `IdentityFile` is the SSH key the gate uses to push that repo
|
||||
upstream after gitleaks passes. The agent itself never holds the
|
||||
upstream credential.
|
||||
|
||||
The Upstream URL is parsed once at construction and the pieces are
|
||||
stashed in the `Upstream*` fields so the git-gate render step
|
||||
doesn't have to re-parse.
|
||||
|
||||
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly
|
||||
one of `identity` (static key path) or `provisioned_key` (automatic
|
||||
lifecycle) must be present. The internal field names are stable."""
|
||||
|
||||
Name: str
|
||||
Upstream: str
|
||||
IdentityFile: str = ""
|
||||
KnownHostKey: str = ""
|
||||
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
|
||||
RemoteKey: str = ""
|
||||
UpstreamUser: str = ""
|
||||
UpstreamHost: str = ""
|
||||
UpstreamPort: str = ""
|
||||
UpstreamPath: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_repos_entry(
|
||||
cls, bottle_name: str, repo_name: str, raw: object
|
||||
) -> "GitEntry":
|
||||
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
||||
|
||||
YAML keys: `url` (required), exactly one of `identity` or
|
||||
`provisioned_key` (required), `host_key` (optional).
|
||||
The repo_name becomes `Name`."""
|
||||
if not repo_name:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.repos has an empty key"
|
||||
)
|
||||
if not _GIT_NAME_RE.match(repo_name):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.repos name {repo_name!r} is invalid; "
|
||||
f"allowed characters: A-Z a-z 0-9 . _ -"
|
||||
)
|
||||
label = f"git-gate.repos[{repo_name!r}]"
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||
for k in d:
|
||||
if k not in {"url", "identity", "provisioned_key", "host_key"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||
f"allowed: url, identity, provisioned_key, host_key"
|
||||
)
|
||||
upstream = d.get("url")
|
||||
if not isinstance(upstream, str) or not upstream:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||
)
|
||||
|
||||
has_identity = "identity" in d
|
||||
has_provisioned = "provisioned_key" in d
|
||||
if has_identity and has_provisioned:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||
f"'identity' or 'provisioned_key'; got both."
|
||||
)
|
||||
if not has_identity and not has_provisioned:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||
f"'identity' or 'provisioned_key'; got neither."
|
||||
)
|
||||
|
||||
ident = ""
|
||||
provisioned_key: Optional[ProvisionedKeyConfig] = None
|
||||
if has_identity:
|
||||
raw_ident = d.get("identity")
|
||||
if not isinstance(raw_ident, str) or not raw_ident:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
|
||||
)
|
||||
ident = raw_ident
|
||||
else:
|
||||
provisioned_key = _parse_provisioned_key_config(
|
||||
bottle_name, label, d["provisioned_key"]
|
||||
)
|
||||
|
||||
khk = _opt_str(
|
||||
d.get("host_key"),
|
||||
f"bottle '{bottle_name}' {label} host_key",
|
||||
)
|
||||
user, host, port, path = parse_git_upstream(
|
||||
upstream, f"bottle '{bottle_name}' {label} url"
|
||||
)
|
||||
return cls(
|
||||
Name=repo_name,
|
||||
Upstream=upstream,
|
||||
IdentityFile=ident,
|
||||
KnownHostKey=khk,
|
||||
ProvisionedKey=provisioned_key,
|
||||
RemoteKey=host,
|
||||
UpstreamUser=user,
|
||||
UpstreamHost=host,
|
||||
UpstreamPort=port,
|
||||
UpstreamPath=path,
|
||||
)
|
||||
|
||||
|
||||
def _parse_provisioned_key_config(
|
||||
bottle_name: str, label: str, raw: object
|
||||
) -> ProvisionedKeyConfig:
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key")
|
||||
for k in d:
|
||||
if k not in {"provider", "token_env", "api_url"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
|
||||
f"allowed: provider, token_env, api_url"
|
||||
)
|
||||
provider = d.get("provider")
|
||||
if not isinstance(provider, str) or not provider:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||
f"string field 'provider'"
|
||||
)
|
||||
token_env = d.get("token_env")
|
||||
if not isinstance(token_env, str) or not token_env:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||
f"string field 'token_env'"
|
||||
)
|
||||
api_url_raw = d.get("api_url", "")
|
||||
if not isinstance(api_url_raw, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string"
|
||||
)
|
||||
return ProvisionedKeyConfig(
|
||||
provider=provider,
|
||||
token_env=token_env,
|
||||
api_url=api_url_raw,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitUser:
|
||||
"""Per-bottle `git config --global user.name` / `user.email`
|
||||
pair (issue #86). The agent's commits inside the bottle are
|
||||
attributed to this identity rather than the agent image's
|
||||
image-baked default (no user, or whatever the image dropped
|
||||
in). Either or both fields can be set independently.
|
||||
|
||||
`from_dict` is forgiving on shape (a single missing field is
|
||||
fine — we just skip that config line at provisioning) but
|
||||
strict on types (string-or-die)."""
|
||||
|
||||
name: str = ""
|
||||
email: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
||||
for k in d.keys():
|
||||
if k not in {"name", "email"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
|
||||
f"allowed: name, email"
|
||||
)
|
||||
name = d.get("name", "")
|
||||
email = d.get("email", "")
|
||||
if not isinstance(name, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user.name must be a string "
|
||||
f"(was {type(name).__name__})"
|
||||
)
|
||||
if not isinstance(email, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user.email must be a string "
|
||||
f"(was {type(email).__name__})"
|
||||
)
|
||||
if not name and not email:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user is set but neither "
|
||||
f"name nor email is non-empty; remove the block or "
|
||||
f"fill at least one field."
|
||||
)
|
||||
return cls(name=name, email=email)
|
||||
|
||||
def is_empty(self) -> bool:
|
||||
return not self.name and not self.email
|
||||
|
||||
|
||||
def parse_git_gate_config(
|
||||
bottle_name: str,
|
||||
raw: object,
|
||||
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||
for k in d.keys():
|
||||
if k not in {"user", "repos"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
||||
f"allowed: user, repos"
|
||||
)
|
||||
|
||||
git_user = (
|
||||
GitUser.from_dict(bottle_name, d["user"])
|
||||
if "user" in d
|
||||
else GitUser()
|
||||
)
|
||||
|
||||
git: tuple[GitEntry, ...] = ()
|
||||
repos_raw = d.get("repos")
|
||||
if repos_raw is not None:
|
||||
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
|
||||
git = tuple(
|
||||
GitEntry.from_repos_entry(bottle_name, name, entry)
|
||||
for name, entry in repos.items()
|
||||
)
|
||||
validate_unique_git_names(bottle_name, git)
|
||||
|
||||
return git, git_user
|
||||
@@ -1,105 +0,0 @@
|
||||
"""Internal per-file Markdown manifest loader."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .log import warn
|
||||
from .manifest_schema import (
|
||||
entity_name_from_path,
|
||||
validate_agent_frontmatter_keys,
|
||||
validate_bottle_frontmatter_keys,
|
||||
)
|
||||
from .yaml_subset import YamlSubsetError, parse_frontmatter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Agent, Bottle
|
||||
|
||||
|
||||
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
|
||||
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
|
||||
not. The manifest format changed in PRD 0011 and we do not want
|
||||
to silently leave the JSON content unused."""
|
||||
from .manifest import ManifestError
|
||||
|
||||
legacy = dir_path / "bot-bottle.json"
|
||||
if legacy.is_file() and not md_dir.exists():
|
||||
raise ManifestError(
|
||||
f"found {legacy} but {md_dir} does not exist. The manifest "
|
||||
f"format changed in PRD 0011 — rewrite the JSON content "
|
||||
f"as per-file Markdown under {md_dir}/bottles/ and "
|
||||
f"{md_dir}/agents/. See README.md for the schema. "
|
||||
f"({label})"
|
||||
)
|
||||
|
||||
|
||||
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
|
||||
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
|
||||
`{name: Bottle}`. Missing dir returns an empty dict."""
|
||||
from .manifest import ManifestError
|
||||
from .manifest_extends import resolve_bottles
|
||||
|
||||
raws: dict[str, dict[str, object]] = {}
|
||||
if not bottles_dir.is_dir():
|
||||
return {}
|
||||
for path in sorted(bottles_dir.glob("*.md")):
|
||||
name = entity_name_from_path(path)
|
||||
if name is None:
|
||||
warn(
|
||||
f"skipping {path}: filename must match "
|
||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||
)
|
||||
continue
|
||||
try:
|
||||
fm, _body = parse_frontmatter(path.read_text())
|
||||
except OSError as e:
|
||||
raise ManifestError(f"could not read {path}: {e}")
|
||||
except YamlSubsetError as e:
|
||||
raise ManifestError(f"{path}: {e}")
|
||||
validate_bottle_frontmatter_keys(path, fm.keys())
|
||||
raws[name] = fm
|
||||
return resolve_bottles(raws)
|
||||
|
||||
|
||||
def load_agents_from_dir(
|
||||
agents_dir: Path,
|
||||
bottle_names: set[str],
|
||||
*,
|
||||
source: str,
|
||||
) -> dict[str, Agent]:
|
||||
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
||||
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
||||
Missing dir returns an empty dict."""
|
||||
from .manifest import Agent, ManifestError
|
||||
|
||||
out: dict[str, Agent] = {}
|
||||
if not agents_dir.is_dir():
|
||||
return out
|
||||
for path in sorted(agents_dir.glob("*.md")):
|
||||
name = entity_name_from_path(path)
|
||||
if name is None:
|
||||
warn(
|
||||
f"skipping {path}: filename must match "
|
||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||
)
|
||||
continue
|
||||
try:
|
||||
fm, body = parse_frontmatter(path.read_text())
|
||||
except OSError as e:
|
||||
raise ManifestError(f"could not read {path}: {e}")
|
||||
except YamlSubsetError as e:
|
||||
raise ManifestError(f"{path}: {e}")
|
||||
validate_agent_frontmatter_keys(path, fm.keys())
|
||||
# Build the dict Agent.from_dict expects. The body becomes
|
||||
# prompt; Claude Code passthrough fields stay in fm and get
|
||||
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
|
||||
agent_dict: dict[str, object] = {
|
||||
"bottle": fm.get("bottle"),
|
||||
"skills": fm.get("skills", []),
|
||||
"prompt": body.strip(),
|
||||
}
|
||||
if "git-gate" in fm:
|
||||
agent_dict["git-gate"] = fm["git-gate"]
|
||||
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
|
||||
return out
|
||||
@@ -1,70 +0,0 @@
|
||||
"""Internal manifest schema policy helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
# Filename-as-key uses kebab-case ASCII. The first character is a
|
||||
# letter so we don't conflict with hidden files / Markdown special
|
||||
# names (`.md`, `_template.md`, etc.). Filenames that fail this
|
||||
# pattern are skipped with a warning rather than crashing the load.
|
||||
_FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
|
||||
|
||||
|
||||
# Frontmatter keys we accept on each entity. Anything not in these
|
||||
# sets dies with a "did you mean" pointer: typos should not silently
|
||||
# ghost into an empty config.
|
||||
BOTTLE_KEYS = frozenset(
|
||||
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
|
||||
)
|
||||
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
|
||||
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
|
||||
|
||||
# Claude Code subagent fields bot-bottle ignores at launch but does
|
||||
# not reject. This lets the same file double as
|
||||
# `~/.claude/agents/*.md` without modification.
|
||||
CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS = frozenset({
|
||||
"name", "description", "model", "color", "memory",
|
||||
})
|
||||
AGENT_KEYS = (
|
||||
AGENT_KEYS_REQUIRED | AGENT_KEYS_OPTIONAL | CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS
|
||||
)
|
||||
AGENT_MODEL_KEYS = AGENT_KEYS | frozenset({"prompt"})
|
||||
|
||||
|
||||
def entity_name_from_path(path: Path) -> str | None:
|
||||
"""Return the entity name implied by the filename, or None if the
|
||||
filename does not fit the [a-z][a-z0-9-]* convention."""
|
||||
if path.suffix != ".md":
|
||||
return None
|
||||
stem = path.stem
|
||||
if not _FILENAME_RX.match(stem):
|
||||
return None
|
||||
return stem
|
||||
|
||||
|
||||
def validate_bottle_frontmatter_keys(path: Path, keys: object) -> None:
|
||||
_validate_frontmatter_keys("bottle", path, keys, BOTTLE_KEYS)
|
||||
|
||||
|
||||
def validate_agent_frontmatter_keys(path: Path, keys: object) -> None:
|
||||
_validate_frontmatter_keys("agent", path, keys, AGENT_KEYS)
|
||||
|
||||
|
||||
def _validate_frontmatter_keys(
|
||||
kind: str,
|
||||
path: Path,
|
||||
keys: object,
|
||||
allowed_keys: frozenset[str],
|
||||
) -> None:
|
||||
from .manifest_util import ManifestError
|
||||
|
||||
key_set = set(keys)
|
||||
unknown = key_set - allowed_keys
|
||||
if unknown:
|
||||
allowed = ", ".join(sorted(allowed_keys))
|
||||
raise ManifestError(
|
||||
f"{kind} file {path}: unknown frontmatter key(s) "
|
||||
f"{sorted(unknown)}; allowed keys are {allowed}."
|
||||
)
|
||||
@@ -1,24 +0,0 @@
|
||||
"""Shared manifest primitives used by all manifest sub-modules."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import cast
|
||||
|
||||
|
||||
class ManifestError(Exception):
|
||||
"""A manifest file (or the manifest tree) is invalid."""
|
||||
|
||||
|
||||
def as_json_object(value: object, label: str) -> dict[str, object]:
|
||||
"""Assert that `value` is a JSON object (str-keyed dict) and return
|
||||
a view typed as `dict[str, object]` so downstream `.get(...)` calls
|
||||
have a typed surface."""
|
||||
if not isinstance(value, dict):
|
||||
raise ManifestError(f"{label} must be a JSON object (was {type(value).__name__})")
|
||||
items = cast(dict[object, object], value)
|
||||
out: dict[str, object] = {}
|
||||
for k, v in items.items():
|
||||
if not isinstance(k, str):
|
||||
raise ManifestError(f"{label} keys must be strings (found {type(k).__name__})")
|
||||
out[k] = v
|
||||
return out
|
||||
+30
-228
@@ -19,8 +19,9 @@ from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
|
||||
from .egress import EGRESS_HOSTNAME, egress_routes_for_bottle
|
||||
from .supervise import SUPERVISE_HOSTNAME
|
||||
from .manifest import Bottle
|
||||
|
||||
@@ -49,17 +50,14 @@ PIPELOCK_HOSTNAME = "pipelock"
|
||||
# --- Allowlist resolution --------------------------------------------------
|
||||
|
||||
|
||||
def pipelock_effective_allowlist(
|
||||
bottle: Bottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> list[str]:
|
||||
def pipelock_effective_allowlist(bottle: Bottle) -> list[str]:
|
||||
"""Hostnames pipelock allows. Sorted for stability.
|
||||
|
||||
Always mirrors `egress_routes_for_bottle(bottle, provider_routes)` —
|
||||
egress is the single allowlist surface, and pipelock's allowlist is
|
||||
the downstream copy for defense-in-depth + DLP body scanning. For
|
||||
bottles without any `egress.routes[]` declared, this is empty except
|
||||
for supervise sidecar traffic when `supervise: true`.
|
||||
Always mirrors `egress_routes_for_bottle(bottle)` — egress is the
|
||||
single allowlist surface, and pipelock's allowlist is the downstream
|
||||
copy for defense-in-depth + DLP body scanning. For bottles without
|
||||
any `egress.routes[]` declared, this is empty except for supervise
|
||||
sidecar traffic when `supervise: true`.
|
||||
|
||||
The supervise sidecar's hostname is auto-added when supervise
|
||||
is enabled (sibling-sidecar traffic that flows through pipelock
|
||||
@@ -67,7 +65,7 @@ def pipelock_effective_allowlist(
|
||||
`bottle.git` do NOT contribute here — git traffic flows
|
||||
through git-gate (PRD 0008), not pipelock."""
|
||||
seen: dict[str, None] = {}
|
||||
for r in egress_routes_for_bottle(bottle, provider_routes):
|
||||
for r in egress_routes_for_bottle(bottle):
|
||||
if r.host:
|
||||
seen.setdefault(r.host, None)
|
||||
if bottle.supervise:
|
||||
@@ -100,23 +98,19 @@ def pipelock_seed_phrase_detection_enabled(bottle: Bottle) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def pipelock_effective_tls_passthrough(
|
||||
bottle: Bottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> list[str]:
|
||||
def pipelock_effective_tls_passthrough(bottle: Bottle) -> list[str]:
|
||||
"""Hostnames pipelock should pass through (no TLS MITM).
|
||||
|
||||
A manifest route opts in with `pipelock.tls_passthrough: true`
|
||||
(lifted into `EgressRoute.tls_passthrough` in `egress_manifest_routes`).
|
||||
Provider routes that set `tls_passthrough=True` (e.g. Codex credential
|
||||
routes where egress injects the host bearer after the agent boundary)
|
||||
are also included. Both arrive via `egress_routes_for_bottle` — no
|
||||
provider-specific branching needed here.
|
||||
A route opts in with `pipelock.tls_passthrough: true`. This is
|
||||
useful for provider API routes where egress injects the
|
||||
Authorization header after the agent boundary; pipelock still
|
||||
enforces the host allowlist but does not decrypt and scan that
|
||||
provider request.
|
||||
"""
|
||||
seen: dict[str, None] = {host: None for host in DEFAULT_TLS_PASSTHROUGH}
|
||||
for route in egress_routes_for_bottle(bottle, provider_routes):
|
||||
if route.tls_passthrough:
|
||||
seen.setdefault(route.host, None)
|
||||
for route in bottle.egress.routes:
|
||||
if route.Pipelock.TlsPassthrough:
|
||||
seen.setdefault(route.Host, None)
|
||||
return sorted(seen.keys())
|
||||
|
||||
|
||||
@@ -148,7 +142,6 @@ def pipelock_build_config(
|
||||
ca_cert_path: str = "",
|
||||
ca_key_path: str = "",
|
||||
ssrf_ip_allowlist: tuple[str, ...] = (),
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> dict[str, object]:
|
||||
"""Build the structured pipelock config dict the sidecar will load.
|
||||
|
||||
@@ -178,7 +171,7 @@ def pipelock_build_config(
|
||||
"version": 1,
|
||||
"mode": "strict",
|
||||
"enforce": True,
|
||||
"api_allowlist": pipelock_effective_allowlist(bottle, provider_routes),
|
||||
"api_allowlist": pipelock_effective_allowlist(bottle),
|
||||
"forward_proxy": {"enabled": True},
|
||||
}
|
||||
if not pipelock_seed_phrase_detection_enabled(bottle):
|
||||
@@ -212,7 +205,7 @@ def pipelock_build_config(
|
||||
"enabled": True,
|
||||
"ca_cert": ca_cert_path,
|
||||
"ca_key": ca_key_path,
|
||||
"passthrough_domains": pipelock_effective_tls_passthrough(bottle, provider_routes),
|
||||
"passthrough_domains": pipelock_effective_tls_passthrough(bottle),
|
||||
}
|
||||
effective_ssrf_ip_allowlist = pipelock_effective_ssrf_ip_allowlist(
|
||||
bottle, ssrf_ip_allowlist,
|
||||
@@ -222,180 +215,6 @@ def pipelock_build_config(
|
||||
return cfg
|
||||
|
||||
|
||||
_PIPELOCK_TOP_LEVEL_KEYS = {
|
||||
"version",
|
||||
"mode",
|
||||
"enforce",
|
||||
"api_allowlist",
|
||||
"seed_phrase_detection",
|
||||
"forward_proxy",
|
||||
"dlp",
|
||||
"request_body_scanning",
|
||||
"tls_interception",
|
||||
"ssrf",
|
||||
}
|
||||
|
||||
|
||||
def _pipelock_render_error(section: str, key: str, expected: str) -> ValueError:
|
||||
return ValueError(
|
||||
f"pipelock_render_yaml: {section}.{key} must be {expected}"
|
||||
)
|
||||
|
||||
|
||||
def _reject_unknown_keys(
|
||||
section: str,
|
||||
obj: dict[str, object],
|
||||
allowed: set[str],
|
||||
) -> None:
|
||||
for key in sorted(set(obj) - allowed):
|
||||
raise ValueError(f"pipelock_render_yaml: {section}.{key} is unsupported")
|
||||
|
||||
|
||||
def _required_dict(
|
||||
obj: dict[str, object],
|
||||
section: str,
|
||||
key: str,
|
||||
) -> dict[str, object]:
|
||||
value = obj.get(key)
|
||||
if not isinstance(value, dict):
|
||||
raise _pipelock_render_error(section, key, "a mapping")
|
||||
return value
|
||||
|
||||
|
||||
def _required_bool(obj: dict[str, object], section: str, key: str) -> bool:
|
||||
value = obj.get(key)
|
||||
if not isinstance(value, bool):
|
||||
raise _pipelock_render_error(section, key, "a boolean")
|
||||
return value
|
||||
|
||||
|
||||
def _required_int(obj: dict[str, object], section: str, key: str) -> int:
|
||||
value = obj.get(key)
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise _pipelock_render_error(section, key, "an integer")
|
||||
return value
|
||||
|
||||
|
||||
def _required_str(obj: dict[str, object], section: str, key: str) -> str:
|
||||
value = obj.get(key)
|
||||
if not isinstance(value, str):
|
||||
raise _pipelock_render_error(section, key, "a string")
|
||||
return value
|
||||
|
||||
|
||||
def _required_str_list(
|
||||
obj: dict[str, object],
|
||||
section: str,
|
||||
key: str,
|
||||
) -> list[str]:
|
||||
value = obj.get(key)
|
||||
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
|
||||
raise _pipelock_render_error(section, key, "a list of strings")
|
||||
return value
|
||||
|
||||
|
||||
def _optional_str_list(
|
||||
obj: dict[str, object],
|
||||
section: str,
|
||||
key: str,
|
||||
) -> list[str]:
|
||||
if key not in obj:
|
||||
return []
|
||||
return _required_str_list(obj, section, key)
|
||||
|
||||
|
||||
def _optional_bool(
|
||||
obj: dict[str, object],
|
||||
section: str,
|
||||
key: str,
|
||||
) -> bool | None:
|
||||
if key not in obj:
|
||||
return None
|
||||
return _required_bool(obj, section, key)
|
||||
|
||||
|
||||
def _optional_str(
|
||||
obj: dict[str, object],
|
||||
section: str,
|
||||
key: str,
|
||||
) -> str | None:
|
||||
if key not in obj:
|
||||
return None
|
||||
return _required_str(obj, section, key)
|
||||
|
||||
|
||||
def _validate_pipelock_render_config(cfg: dict[str, object]) -> dict[str, object]:
|
||||
_reject_unknown_keys("config", cfg, _PIPELOCK_TOP_LEVEL_KEYS)
|
||||
normalized: dict[str, object] = {
|
||||
"version": _required_int(cfg, "config", "version"),
|
||||
"mode": _required_str(cfg, "config", "mode"),
|
||||
"enforce": _required_bool(cfg, "config", "enforce"),
|
||||
"api_allowlist": _required_str_list(cfg, "config", "api_allowlist"),
|
||||
}
|
||||
|
||||
if "seed_phrase_detection" in cfg:
|
||||
spd = _required_dict(cfg, "config", "seed_phrase_detection")
|
||||
_reject_unknown_keys("seed_phrase_detection", spd, {"enabled"})
|
||||
normalized["seed_phrase_detection"] = {
|
||||
"enabled": _required_bool(spd, "seed_phrase_detection", "enabled"),
|
||||
}
|
||||
|
||||
fp = _required_dict(cfg, "config", "forward_proxy")
|
||||
_reject_unknown_keys("forward_proxy", fp, {"enabled"})
|
||||
normalized["forward_proxy"] = {
|
||||
"enabled": _required_bool(fp, "forward_proxy", "enabled"),
|
||||
}
|
||||
|
||||
dlp = _required_dict(cfg, "config", "dlp")
|
||||
_reject_unknown_keys("dlp", dlp, {"include_defaults", "scan_env"})
|
||||
normalized["dlp"] = {
|
||||
"include_defaults": _required_bool(dlp, "dlp", "include_defaults"),
|
||||
"scan_env": _required_bool(dlp, "dlp", "scan_env"),
|
||||
}
|
||||
|
||||
rbs = _required_dict(cfg, "config", "request_body_scanning")
|
||||
_reject_unknown_keys(
|
||||
"request_body_scanning",
|
||||
rbs,
|
||||
{"action", "scan_headers", "header_mode"},
|
||||
)
|
||||
normalized_rbs: dict[str, object] = {
|
||||
"action": _required_str(rbs, "request_body_scanning", "action"),
|
||||
}
|
||||
scan_headers = _optional_bool(rbs, "request_body_scanning", "scan_headers")
|
||||
if scan_headers is not None:
|
||||
normalized_rbs["scan_headers"] = scan_headers
|
||||
header_mode = _optional_str(rbs, "request_body_scanning", "header_mode")
|
||||
if header_mode is not None:
|
||||
normalized_rbs["header_mode"] = header_mode
|
||||
normalized["request_body_scanning"] = normalized_rbs
|
||||
|
||||
if "tls_interception" in cfg:
|
||||
tls = _required_dict(cfg, "config", "tls_interception")
|
||||
_reject_unknown_keys(
|
||||
"tls_interception",
|
||||
tls,
|
||||
{"enabled", "ca_cert", "ca_key", "passthrough_domains"},
|
||||
)
|
||||
normalized["tls_interception"] = {
|
||||
"enabled": _required_bool(tls, "tls_interception", "enabled"),
|
||||
"ca_cert": _required_str(tls, "tls_interception", "ca_cert"),
|
||||
"ca_key": _required_str(tls, "tls_interception", "ca_key"),
|
||||
"passthrough_domains": _optional_str_list(
|
||||
tls, "tls_interception", "passthrough_domains",
|
||||
),
|
||||
}
|
||||
|
||||
if "ssrf" in cfg:
|
||||
ssrf = _required_dict(cfg, "config", "ssrf")
|
||||
_reject_unknown_keys("ssrf", ssrf, {"ip_allowlist"})
|
||||
normalized["ssrf"] = {
|
||||
"ip_allowlist": _required_str_list(ssrf, "ssrf", "ip_allowlist"),
|
||||
}
|
||||
|
||||
return normalized
|
||||
|
||||
|
||||
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||
"""Render a pipelock config dict (as produced by
|
||||
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
||||
@@ -403,38 +222,31 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||
def _bool(b: object) -> str:
|
||||
return "true" if b else "false"
|
||||
|
||||
cfg = _validate_pipelock_render_config(cfg)
|
||||
lines: list[str] = []
|
||||
lines.append(f"version: {cfg['version']}")
|
||||
lines.append(f"mode: {cfg['mode']}")
|
||||
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
||||
lines.append("")
|
||||
lines.append("api_allowlist:")
|
||||
api_allowlist = cfg["api_allowlist"]
|
||||
assert isinstance(api_allowlist, list)
|
||||
for h in api_allowlist:
|
||||
for h in cast(list[str], cfg["api_allowlist"]):
|
||||
lines.append(f' - "{h}"')
|
||||
lines.append("")
|
||||
if "seed_phrase_detection" in cfg:
|
||||
lines.append("seed_phrase_detection:")
|
||||
spd = cfg["seed_phrase_detection"]
|
||||
assert isinstance(spd, dict)
|
||||
spd = cast(dict[str, object], cfg["seed_phrase_detection"])
|
||||
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
||||
lines.append("")
|
||||
lines.append("forward_proxy:")
|
||||
fp = cfg["forward_proxy"]
|
||||
assert isinstance(fp, dict)
|
||||
fp = cast(dict[str, object], cfg["forward_proxy"])
|
||||
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
||||
lines.append("")
|
||||
lines.append("dlp:")
|
||||
dlp = cfg["dlp"]
|
||||
assert isinstance(dlp, dict)
|
||||
dlp = cast(dict[str, object], cfg["dlp"])
|
||||
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
||||
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
||||
lines.append("")
|
||||
lines.append("request_body_scanning:")
|
||||
rbs = cfg["request_body_scanning"]
|
||||
assert isinstance(rbs, dict)
|
||||
rbs = cast(dict[str, object], cfg["request_body_scanning"])
|
||||
lines.append(f' action: "{rbs["action"]}"')
|
||||
if "scan_headers" in rbs:
|
||||
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
||||
@@ -443,13 +255,11 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||
if "tls_interception" in cfg:
|
||||
lines.append("")
|
||||
lines.append("tls_interception:")
|
||||
tls = cfg["tls_interception"]
|
||||
assert isinstance(tls, dict)
|
||||
tls = cast(dict[str, object], cfg["tls_interception"])
|
||||
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
||||
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
||||
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
||||
passthrough = tls["passthrough_domains"]
|
||||
assert isinstance(passthrough, list)
|
||||
passthrough = cast(list[str], tls.get("passthrough_domains", []))
|
||||
if passthrough:
|
||||
lines.append(" passthrough_domains:")
|
||||
for d in passthrough:
|
||||
@@ -457,12 +267,9 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||
if "ssrf" in cfg:
|
||||
lines.append("")
|
||||
lines.append("ssrf:")
|
||||
ssrf = cfg["ssrf"]
|
||||
assert isinstance(ssrf, dict)
|
||||
ssrf = cast(dict[str, object], cfg["ssrf"])
|
||||
lines.append(" ip_allowlist:")
|
||||
ip_allowlist = ssrf["ip_allowlist"]
|
||||
assert isinstance(ip_allowlist, list)
|
||||
for ip in ip_allowlist:
|
||||
for ip in cast(list[str], ssrf["ip_allowlist"]):
|
||||
lines.append(f' - "{ip}"')
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
@@ -512,11 +319,7 @@ class PipelockProxy:
|
||||
(`PIPELOCK_CA_CERT_IN_CONTAINER` / `PIPELOCK_CA_KEY_IN_CONTAINER`)."""
|
||||
|
||||
def prepare(
|
||||
self,
|
||||
bottle: Bottle,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
self, bottle: Bottle, slug: str, stage_dir: Path
|
||||
) -> PipelockProxyPlan:
|
||||
"""Write the pipelock yaml config (mode 600) under `stage_dir`
|
||||
and return the plan for launch. Pure host-side, no docker
|
||||
@@ -539,7 +342,6 @@ class PipelockProxy:
|
||||
bottle,
|
||||
ca_cert_path=PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||
ca_key_path=PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||
provider_routes=provider_routes,
|
||||
)
|
||||
yaml_path.write_text(pipelock_render_yaml(cfg))
|
||||
yaml_path.chmod(0o600)
|
||||
|
||||
@@ -20,7 +20,7 @@ sick daemon."
|
||||
|
||||
Daemon subset is env-driven. The compose renderer narrows it via
|
||||
`BOT_BOTTLE_SIDECAR_DAEMONS=egress,pipelock` for bottles that
|
||||
don't use git-gate or supervise. Default: all daemons.
|
||||
don't use git-gate or supervise. Default: all four.
|
||||
|
||||
Stdlib-only by design — adding supervisord/s6/runit for four
|
||||
daemons is heavier than this script.
|
||||
@@ -98,7 +98,6 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
|
||||
"--listen", "0.0.0.0:8888"),
|
||||
),
|
||||
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
|
||||
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")),
|
||||
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
|
||||
)
|
||||
|
||||
@@ -163,10 +162,6 @@ class _Supervisor:
|
||||
# Names of children that have been logged as having exited
|
||||
# so we only log each death once across watch-loop ticks.
|
||||
self._logged_dead: set[str] = set()
|
||||
# Signal handlers add daemon names here and return quickly.
|
||||
# The main watch loop drains the set, so repeated restart
|
||||
# requests for one daemon coalesce into one restart.
|
||||
self._restart_requested: set[str] = set()
|
||||
|
||||
def start_all(self) -> None:
|
||||
for spec in self.specs:
|
||||
@@ -177,7 +172,6 @@ class _Supervisor:
|
||||
if self.shutdown_at is not None:
|
||||
return
|
||||
self.shutdown_at = time.monotonic()
|
||||
self._restart_requested.clear()
|
||||
_log(f"shutting down ({reason}); forwarding SIGTERM")
|
||||
for _, p in self.procs:
|
||||
if p.poll() is None:
|
||||
@@ -186,24 +180,6 @@ class _Supervisor:
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
|
||||
def request_restart(self, daemon_name: str) -> bool:
|
||||
"""Queue a daemon restart for the main loop to process.
|
||||
|
||||
Signal handlers use this non-blocking path instead of doing
|
||||
subprocess lifecycle work directly. Requests coalesce by
|
||||
daemon name: one pending restart is enough to make the daemon
|
||||
reread the latest config from disk.
|
||||
|
||||
Returns True iff a daemon by that name is known to the
|
||||
supervisor and shutdown has not started."""
|
||||
if self.shutdown_at is not None:
|
||||
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
|
||||
return False
|
||||
if not any(spec.name == daemon_name for spec, _ in self.procs):
|
||||
return False
|
||||
self._restart_requested.add(daemon_name)
|
||||
return True
|
||||
|
||||
def tick(self) -> bool:
|
||||
"""One iteration of the watch loop. Returns True when every
|
||||
child has exited and the supervisor can return.
|
||||
@@ -211,8 +187,6 @@ class _Supervisor:
|
||||
A child dying unexpectedly is logged but does NOT initiate
|
||||
shutdown — see the module docstring's failure-policy
|
||||
section. Shutdown is signal-driven only."""
|
||||
self._drain_restart_requests()
|
||||
|
||||
for spec, p in self.procs:
|
||||
rc = p.poll()
|
||||
if rc is None or spec.name in self._logged_dead:
|
||||
@@ -245,37 +219,14 @@ class _Supervisor:
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
|
||||
done = all(p.poll() is not None for _, p in self.procs)
|
||||
if done:
|
||||
for _, p in self.procs:
|
||||
if p.stdout is not None:
|
||||
p.stdout.close()
|
||||
return done
|
||||
return all(p.poll() is not None for _, p in self.procs)
|
||||
|
||||
def exit_code(self) -> int:
|
||||
"""Positive child failures win; otherwise report success.
|
||||
|
||||
Python represents signal-terminated children as negative
|
||||
return codes. A signal-only graceful shutdown should not leak
|
||||
that platform-specific detail into the container exit status,
|
||||
but a positive crash before shutdown should remain visible."""
|
||||
positives = [
|
||||
p.returncode for _, p in self.procs
|
||||
if p.returncode is not None and p.returncode > 0
|
||||
]
|
||||
return max(positives, default=0)
|
||||
|
||||
def _drain_restart_requests(self) -> None:
|
||||
if self.shutdown_at is not None:
|
||||
self._restart_requested.clear()
|
||||
return
|
||||
requested = tuple(sorted(self._restart_requested))
|
||||
self._restart_requested.clear()
|
||||
for daemon_name in requested:
|
||||
if self.shutdown_at is not None:
|
||||
self._restart_requested.clear()
|
||||
return
|
||||
self.restart_daemon(daemon_name)
|
||||
"""Worst child returncode wins. On graceful shutdown every
|
||||
child is signal-killed (negative returncode) and max()
|
||||
returns 0; if some child crashed nonzero before the signal
|
||||
the operator gets that code on container exit."""
|
||||
return max((p.returncode for _, p in self.procs), default=0)
|
||||
|
||||
def forward_signal(self, sig: int, daemon_name: str) -> bool:
|
||||
"""Forward a signal to one named child. Used by the SIGHUP
|
||||
@@ -340,8 +291,6 @@ class _Supervisor:
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
p.wait()
|
||||
if p.stdout is not None:
|
||||
p.stdout.close()
|
||||
self._logged_dead.discard(daemon_name)
|
||||
new_proc = _spawn(spec)
|
||||
self.procs[idx] = (spec, new_proc)
|
||||
@@ -373,7 +322,7 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
# supervisor restarts the pipelock daemon in place (other
|
||||
# daemons keep running — specifically supervise, whose MCP
|
||||
# socket would drop on a whole-container `docker restart`).
|
||||
signal.signal(signal.SIGUSR1, lambda *_: sup.request_restart("pipelock"))
|
||||
signal.signal(signal.SIGUSR1, lambda *_: sup.restart_daemon("pipelock"))
|
||||
|
||||
while not sup.tick():
|
||||
time.sleep(_POLL_INTERVAL)
|
||||
|
||||
@@ -12,8 +12,8 @@ agent calls when it hits a stuck-recovery category:
|
||||
Each tool call: the agent passes the full proposed file plus a
|
||||
justification text. The sidecar validates the proposal syntactically,
|
||||
writes it to the host's per-bottle queue dir, and holds the tool-call
|
||||
connection open. The operator's supervise TUI
|
||||
(bot_bottle.cli.supervise) sees the proposal, accepts
|
||||
connection open. The operator's TUI dashboard
|
||||
(bot_bottle.cli.dashboard) sees the proposal, accepts
|
||||
approve / modify / reject, and writes a response file alongside the
|
||||
proposal. The sidecar sees the response and returns `{status, notes}`
|
||||
to the agent.
|
||||
|
||||
@@ -35,7 +35,6 @@ import json
|
||||
import os
|
||||
import socketserver
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
@@ -64,10 +63,6 @@ ERR_METHOD_NOT_FOUND = -32601
|
||||
ERR_INVALID_PARAMS = -32602
|
||||
ERR_INTERNAL = -32603
|
||||
|
||||
DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
|
||||
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
|
||||
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class JsonRpcRequest:
|
||||
@@ -417,7 +412,6 @@ def _validate_and_bundle_egress_route(
|
||||
class ServerConfig:
|
||||
bottle_slug: str
|
||||
queue_dir: Path
|
||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
||||
|
||||
|
||||
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
|
||||
@@ -448,7 +442,7 @@ def handle_list_egress_routes(
|
||||
})
|
||||
opener = urllib.request.build_opener(proxy_handler)
|
||||
try:
|
||||
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=EGRESS_LIST_TIMEOUT_SECONDS) as resp:
|
||||
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=5) as resp:
|
||||
body = resp.read().decode("utf-8")
|
||||
except (urllib.error.URLError, OSError) as e:
|
||||
return {
|
||||
@@ -526,20 +520,7 @@ def handle_tools_call(
|
||||
f"for bottle {config.bottle_slug}; waiting for operator...\n"
|
||||
)
|
||||
sys.stderr.flush()
|
||||
deadline = time.monotonic() + config.response_timeout_seconds
|
||||
try:
|
||||
response = _sv.wait_for_response(
|
||||
config.queue_dir,
|
||||
proposal.id,
|
||||
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
|
||||
deadline=deadline,
|
||||
)
|
||||
except TimeoutError:
|
||||
text = format_pending_response_text(config.response_timeout_seconds)
|
||||
return {
|
||||
"content": [{"type": "text", "text": text}],
|
||||
"isError": False,
|
||||
}
|
||||
response = _sv.wait_for_response(config.queue_dir, proposal.id)
|
||||
_sv.archive_proposal(config.queue_dir, proposal.id)
|
||||
|
||||
text = format_response_text(response)
|
||||
@@ -561,16 +542,6 @@ def format_response_text(response: "_sv.Response") -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def format_pending_response_text(timeout_seconds: float) -> str:
|
||||
return "\n".join([
|
||||
"status: pending",
|
||||
(
|
||||
"notes: operator response timed out after "
|
||||
f"{timeout_seconds:g}s; proposal remains queued"
|
||||
),
|
||||
])
|
||||
|
||||
|
||||
# --- HTTP transport --------------------------------------------------------
|
||||
|
||||
|
||||
@@ -683,15 +654,10 @@ def serve(
|
||||
queue_dir: Path,
|
||||
port: int = _sv.SUPERVISE_PORT,
|
||||
bind: str = "0.0.0.0",
|
||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
|
||||
) -> typing.NoReturn:
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
server = MCPServer((bind, port), MCPHandler)
|
||||
server.config = ServerConfig(
|
||||
bottle_slug=bottle_slug,
|
||||
queue_dir=queue_dir,
|
||||
response_timeout_seconds=response_timeout_seconds,
|
||||
)
|
||||
server.config = ServerConfig(bottle_slug=bottle_slug, queue_dir=queue_dir)
|
||||
sys.stderr.write(
|
||||
f"supervise listening on {bind}:{port}; "
|
||||
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
||||
@@ -716,37 +682,9 @@ def main(argv: list[str]) -> int:
|
||||
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
||||
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
||||
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
||||
try:
|
||||
response_timeout_seconds = _response_timeout_from_env(os.environ)
|
||||
except ValueError as e:
|
||||
sys.stderr.write(f"supervise: {e}\n")
|
||||
return 2
|
||||
serve(
|
||||
bottle_slug=bottle_slug,
|
||||
queue_dir=queue_dir,
|
||||
port=port,
|
||||
bind=bind,
|
||||
response_timeout_seconds=response_timeout_seconds,
|
||||
)
|
||||
serve(bottle_slug=bottle_slug, queue_dir=queue_dir, port=port, bind=bind)
|
||||
return 0 # serve() does not return
|
||||
|
||||
|
||||
def _response_timeout_from_env(env: typing.Mapping[str, str]) -> float:
|
||||
raw = env.get("SUPERVISE_RESPONSE_TIMEOUT_SECONDS", "").strip()
|
||||
if not raw:
|
||||
return DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
||||
try:
|
||||
value = float(raw)
|
||||
except ValueError as e:
|
||||
raise ValueError(
|
||||
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
||||
) from e
|
||||
if value <= 0:
|
||||
raise ValueError(
|
||||
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
|
||||
@@ -5,18 +5,9 @@ level deeper, under their backend package."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import os
|
||||
|
||||
|
||||
def is_ip_literal(value: str) -> bool:
|
||||
try:
|
||||
ipaddress.ip_address(value)
|
||||
except ValueError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def expand_tilde(path: str) -> str:
|
||||
"""Expand a leading '~' to $HOME. Leaves paths without a leading
|
||||
tilde unchanged. Falls back to the empty string if $HOME is unset
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
"""Backend-neutral plan for porting the operator workspace."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
|
||||
WORKSPACE_DIRNAME = "workspace"
|
||||
DEFAULT_WORKSPACE_OWNER = "node:node"
|
||||
DEFAULT_WORKSPACE_MODE = "755"
|
||||
|
||||
|
||||
class WorkspaceSpec(Protocol):
|
||||
copy_cwd: bool
|
||||
user_cwd: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkspacePlan:
|
||||
"""Resolved workspace contract shared by all bottle backends."""
|
||||
|
||||
enabled: bool
|
||||
host_path: Path
|
||||
guest_home: str
|
||||
guest_path: str
|
||||
workdir: str
|
||||
owner: str = DEFAULT_WORKSPACE_OWNER
|
||||
mode: str = DEFAULT_WORKSPACE_MODE
|
||||
copy_contents: bool = True
|
||||
copy_git: bool = True
|
||||
has_host_git_dir: bool = False
|
||||
|
||||
|
||||
def workspace_plan(spec: WorkspaceSpec, *, guest_home: str) -> WorkspacePlan:
|
||||
"""Resolve the in-bottle workspace path from CLI intent."""
|
||||
host_path = Path(spec.user_cwd).expanduser()
|
||||
if spec.copy_cwd:
|
||||
guest_path = f"{guest_home.rstrip('/')}/{WORKSPACE_DIRNAME}"
|
||||
workdir = guest_path
|
||||
else:
|
||||
guest_path = guest_home
|
||||
workdir = guest_home
|
||||
return WorkspacePlan(
|
||||
enabled=spec.copy_cwd,
|
||||
host_path=host_path,
|
||||
guest_home=guest_home,
|
||||
guest_path=guest_path,
|
||||
workdir=workdir,
|
||||
has_host_git_dir=(host_path / ".git").is_dir(),
|
||||
)
|
||||
@@ -1,43 +0,0 @@
|
||||
# ADR 0003: Keep agent system prompts user-directed, not auto-generated from config
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-05-29
|
||||
- **Deciders:** didericis
|
||||
|
||||
## Context
|
||||
|
||||
A bottle already declares exactly what an agent can reach: egress routes
|
||||
(allowlisted hosts + auth) and git config (remotes + identity). We
|
||||
considered deriving an agent's system prompt — or a section of it —
|
||||
automatically from those configs, so an agent would be told up front
|
||||
what it has access to (e.g. "you can reach `gitea.dideric.is` over the
|
||||
git remote and its API"). The question surfaced while hand-writing that
|
||||
exact line into the `claude-implementer` prompt.
|
||||
|
||||
## Decision
|
||||
|
||||
System prompts stay **user-directed** — authored by the operator. We do
|
||||
not auto-generate prompt content from a bottle's egress / git config.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The operator controls what the agent is *told* about its environment,
|
||||
independently of what the bottle *grants*. Sometimes we may want to
|
||||
withhold that information from the agent directly — keep the prompt
|
||||
silent about an allowlisted host even though egress permits it.
|
||||
- The agent can still infer its access on its own (attempt a request,
|
||||
read its env, `git remote -v`, the gitconfig), so auto-injection is a
|
||||
convenience, not a capability the agent depends on.
|
||||
- Cost accepted: operators must restate access in the prompt when they
|
||||
want the agent to know it (as we did for the Gitea instance), and the
|
||||
prompt can drift from the config. That decoupling of "what the bottle
|
||||
grants" from "what the agent is told" is the point.
|
||||
- Revisit if keeping prompts in sync with configs becomes a real pain.
|
||||
An *opt-in* helper that emits a capability summary the operator
|
||||
chooses to include would honor this decision; silent auto-injection
|
||||
would not.
|
||||
|
||||
## Links
|
||||
|
||||
- ADR 0002 (`0002-agent-identity-claimed-not-vouched.md`) — related
|
||||
agent-trust posture (what the agent is granted vs. what it can claim).
|
||||
@@ -3,8 +3,8 @@
|
||||
Short, durable records of decisions — one file per decision. This is a
|
||||
lightweight [Architecture Decision Record](https://adr.github.io/)
|
||||
practice: capture *what was decided and why* in a versioned file so the
|
||||
reasoning lives in the clone, not in a Gitea issue thread or a chat log
|
||||
that disappears when the host does.
|
||||
reasoning lives in the clone, not in a forge issue thread or a chat log
|
||||
that disappears when the provider does.
|
||||
|
||||
See `docs/research/issue-tracking-vs-in-repo-decision-history.md` for
|
||||
the rationale behind keeping decision history in-repo, and
|
||||
@@ -34,7 +34,7 @@ What we decided, stated plainly.
|
||||
What follows — the good, and the costs/trade-offs accepted.
|
||||
|
||||
## Links
|
||||
PRDs, research notes, issues/PRs. Gitea links are convenience
|
||||
PRDs, research notes, issues/PRs. Forge links are convenience
|
||||
pointers; the reasoning above must stand without them.
|
||||
```
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0001: Per-agent egress proxy via pipelock
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-08
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0002: Test pipeline on Gitea Actions
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-08
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0003: Bottle Backend abstraction
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-10
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0004: Split out provisioners
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-11
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0006: pipelock native TLS interception
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-12
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0008: Git gate
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-12
|
||||
|
||||
@@ -83,7 +83,12 @@ for a declared upstream:
|
||||
- **Manifest field.** `bottle.git` — a list of git remotes the
|
||||
bottle is allowed to talk to, each with the credential the gate
|
||||
uses to push upstream. The agent gets no parallel `bottle.ssh`
|
||||
entry for those upstreams.
|
||||
entry for those upstreams. Each entry may also carry an
|
||||
`ExtraHosts: { hostname: ip }` map, surfaced to the gate as
|
||||
`--add-host` so the gate can resolve upstreams whose public DNS
|
||||
doesn't point at the reachable IP (e.g. Tailscale-only hosts).
|
||||
The agent-side `insteadOf` rewrite keys off the original hostname,
|
||||
so the manifest's `Upstream` URL stays human-readable.
|
||||
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
|
||||
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
|
||||
operation against the declared upstream (push, fetch, clone,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0009: Remove ssh-gate and bottle.ssh
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-13
|
||||
|
||||
@@ -88,7 +88,8 @@ the unused path.
|
||||
- **Pipelock interaction.** Drop the SSH-derived branch from
|
||||
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
|
||||
there is no per-upstream IP carve-out to render; git-gate
|
||||
has its own egress network.
|
||||
has its own egress network and pulls in upstream resolution
|
||||
via `ExtraHosts` plus DNS.
|
||||
- **Tests.** Delete the ssh-gate unit + integration suites,
|
||||
the ssh fixtures in `tests/fixtures.py`, and the
|
||||
shadow-route assertions in `test_manifest_git.py`. Adjust
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0011: Per-file Markdown manifest
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-24
|
||||
|
||||
@@ -274,6 +274,8 @@ git:
|
||||
Name: bot-bottle
|
||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
||||
ExtraHosts:
|
||||
gitea.dideric.is: 100.78.141.42
|
||||
KnownHostKey: ssh-rsa AAAAB3...
|
||||
egress:
|
||||
allowlist:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0012: Stuck-agent recovery flow
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-24
|
||||
|
||||
@@ -22,7 +22,7 @@ A real stuck agent recovers end-to-end in each of the three categories: a **cred
|
||||
|
||||
- Live attach or in-place mutation of running containers. The whole design exists to avoid this.
|
||||
- Agent-to-agent communication. Re-stated from the project's existing non-goals; the recovery flow is human→agent only.
|
||||
- Auditing or forensic replay of agent runs. Git/Gitea history is the audit log; this PRD does not add a separate run log.
|
||||
- Auditing or forensic replay of agent runs. Git/forge history is the audit log; this PRD does not add a separate run log.
|
||||
- Reducing time-to-unstuck below some target. Faster than kill-and-restart is implicit, but no specific SLO is in scope.
|
||||
|
||||
## Stuck categories
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0013: Supervise plane foundation
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-25
|
||||
- **Parent:** PRD 0012
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0015: pipelock block remediation
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-25
|
||||
- **Parent:** PRD 0012
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0016: capability block remediation
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-25
|
||||
- **Parent:** PRD 0012
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0017: Egress-proxy — universal MITM with path filtering + auth injection
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-25
|
||||
- **Supersedes:** the cred-proxy sidecar (PRD 0010) — hard cutover.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0018: One Compose project per bottle instance
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-25
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0019: Active agents in the dashboard, agent-scoped edit verbs
|
||||
|
||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0020: Start and attach to agents from inside the dashboard
|
||||
|
||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0021: Dashboard as left tmux pane, selected agent as right pane
|
||||
|
||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0022: End-to-end sandbox-escape integration test
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0023: smolmachines bottle backend
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0024: Consolidate per-bottle sidecars into a single bundle
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-26
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0025: Bottle composition via `extends:`
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-27
|
||||
- **Issue:** #88
|
||||
@@ -161,7 +161,8 @@ expectation. (Same model as shell `export` precedence.)
|
||||
`git.remotes` is also keyed, so it follows dict-style inheritance:
|
||||
children can override one host without restating every remote. The
|
||||
remote entry is replaced as a whole on host collision because
|
||||
`Upstream`, `IdentityFile`, and `KnownHostKey` are tightly coupled.
|
||||
`Upstream`, `IdentityFile`, `KnownHostKey`, and `ExtraHosts` are
|
||||
tightly coupled.
|
||||
|
||||
The `git.user` dataclass-overlay (each non-empty field wins
|
||||
individually) is so a parent can declare `git.user.name` and a
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PRD 0026: Agent Provider Templates
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Draft
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-05-28
|
||||
|
||||
@@ -86,10 +86,7 @@ agent_provider:
|
||||
|
||||
## Open questions
|
||||
|
||||
- `codex_auth` is retained as a placeholder marker for follow-up Codex
|
||||
credential-injection work. The Codex template should not inject an
|
||||
`OPENAI_API_KEY` placeholder; Codex bottles use device/ChatGPT login
|
||||
state instead.
|
||||
- The initial Codex auth role is `codex_auth`; it provides a non-secret `OPENAI_API_KEY` placeholder to the agent while egress holds the real token.
|
||||
- Existing state-folder transcript capture is Claude-specific and should remain gated to Claude until the follow-up state/transcript refactor.
|
||||
|
||||
## References
|
||||
|
||||
@@ -1,226 +0,0 @@
|
||||
# PRD 0027: Agent-level git user identity
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-28
|
||||
- **Issue:** #94
|
||||
|
||||
## Summary
|
||||
|
||||
Let an **agent** file declare `git.user` (name / email). At launch
|
||||
the agent's `git.user` overlays the referenced bottle's `git.user`
|
||||
per-field (agent wins on non-empty fields), mirroring the
|
||||
`extends:` overlay from PRD 0025. `git.remotes` stays bottle-only.
|
||||
|
||||
Solves the "I need a whole separate bottle just to change the commit
|
||||
name" coupling: today commit attribution — a purpose/presentation
|
||||
concern — can only be varied by authoring a new security boundary
|
||||
(a bottle). After this change, two agents (e.g. `implementer` and
|
||||
`reviewer`) can share one bottle and still commit under distinct
|
||||
identities.
|
||||
|
||||
## Problem
|
||||
|
||||
`git.user.name` / `git.user.email` is a bottle-only field. Agent
|
||||
frontmatter is validated against a strict allowlist (`_AGENT_KEYS`
|
||||
in `manifest.py`); a `git:` block in an agent file dies at parse
|
||||
with "unknown frontmatter key(s)". So the only way to give an agent
|
||||
a distinct commit identity is to give it a distinct *bottle*.
|
||||
|
||||
That couples identity to the trust boundary. Bottles are home-only
|
||||
and define security (egress, credentials, provider). Forcing a new
|
||||
bottle per commit-name means either (a) duplicating a bottle just to
|
||||
flip one string, or (b) leaning on PRD 0025 `extends:` to make a
|
||||
near-clone whose only delta is `git.user`. Both are heavier than the
|
||||
concern deserves.
|
||||
|
||||
## Why this is safe (trust analysis)
|
||||
|
||||
Allowing agent files — which can live in `$CWD/.bot-bottle/agents/`
|
||||
and therefore be supplied by a cloned repo — to set `git.user` does
|
||||
**not** weaken the security model, because git author identity is
|
||||
not a credential or a capability:
|
||||
|
||||
- **Push auth is separate.** Authentication to a remote is the
|
||||
bottle's `git.remotes` IdentityFile / token. `user.name` /
|
||||
`user.email` grant zero access.
|
||||
- **It's already forgeable.** The agent inside the bottle can run
|
||||
`git config user.email <anything>` or `git commit --author=...`
|
||||
at runtime regardless of the manifest. The manifest field is only
|
||||
a *default*; allowing agents to set it adds no capability that
|
||||
isn't already reachable one layer down.
|
||||
- **Authorship was never a trust anchor.** If attribution integrity
|
||||
matters, that is a commit-*signing* concern (SSH/GPG), which a
|
||||
name/email field cannot provide either way.
|
||||
|
||||
The one residual is cosmetic impersonation (a cloned repo's agent
|
||||
file could set an identity that reads like a real person's). Commits
|
||||
still push under the bottle's credentials, and the author string was
|
||||
never vouched-for, so this is presentation, not escalation. We
|
||||
document that an agent identity is *claimed, not vouched*.
|
||||
|
||||
`git.remotes` is explicitly **not** lifted to the agent layer — that
|
||||
block carries credentials and host trust (IdentityFile, KnownHostKey)
|
||||
and stays a bottle-only, home-only concern.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- Add an optional `git.user` block to the agent frontmatter schema
|
||||
(name and/or email), reusing the existing `GitUser` validator.
|
||||
- An agent's declared `git.user` overlays the referenced bottle's
|
||||
`git.user` per-field; each non-empty agent field wins, empties
|
||||
fall through to the bottle. Identical overlay semantics to the
|
||||
bottle `extends:` merge (PRD 0025).
|
||||
- `git.remotes` in an agent file is rejected at parse with a clear
|
||||
"bottle-only" pointer.
|
||||
- The overlay is applied at `Manifest.bottle_for()` — the single
|
||||
point both backends call to resolve an agent's bottle — so the
|
||||
docker and smolmachines provisioners need no changes.
|
||||
- Existing agent files continue to parse identically — `git.user`
|
||||
is opt-in.
|
||||
- **Identity provenance is surfaced.** The y/N preflight (both
|
||||
backends) and `cli.py info <agent>` print the effective git
|
||||
identity with a per-field `(agent)` / `(bottle)` annotation so the
|
||||
operator can see which level each field came from.
|
||||
- **The example collapses to demonstrate the pattern.** The bundled
|
||||
example replaces the identity-only-bottle shape with one shared
|
||||
bottle + per-agent `git.user`, showing the intended usage.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **No agent-level `git.remotes`.** Credentials and host trust stay
|
||||
bottle-only and home-only.
|
||||
- **No other agent-level bottle fields.** This PRD lifts `git.user`
|
||||
and nothing else; agents do not gain `egress`, `env`,
|
||||
`agent_provider`, etc. (that is the issue #88 design PRD 0025
|
||||
deliberately rejected).
|
||||
- **No commit signing.** Attribution integrity via SSH/GPG is a
|
||||
separate concern, out of scope here.
|
||||
- **No CWD-vs-HOME identity gating.** Because the field is
|
||||
non-enforcing (see trust analysis), CWD and HOME agents are
|
||||
treated the same. If a future change makes identity load-bearing,
|
||||
revisit.
|
||||
|
||||
## Design
|
||||
|
||||
### Schema
|
||||
|
||||
A new optional `git:` block on agent files, carrying only `user`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
bottle: claude-dev
|
||||
git:
|
||||
user:
|
||||
name: claude-implementer
|
||||
email: eric+claude-implementer@dideric.is
|
||||
---
|
||||
|
||||
You are a feature-implementation agent ...
|
||||
```
|
||||
|
||||
- `git.user` accepts `name` and/or `email` (string-or-die; at least
|
||||
one non-empty), validated by the **existing** `GitUser.from_dict`.
|
||||
- `git.remotes` (or any `git` key other than `user`) in an agent
|
||||
file dies at parse: "git.remotes is bottle-only".
|
||||
- `git` is added to `_AGENT_KEYS`; the agent md-loader threads the
|
||||
raw `git` block into the dict `Agent.from_dict` consumes.
|
||||
|
||||
### Merge rule
|
||||
|
||||
When `Manifest.bottle_for(agent)` resolves the agent's bottle, it
|
||||
returns the bottle with `git_user` overlaid by the agent's
|
||||
`git_user`:
|
||||
|
||||
| Field | Merge |
|
||||
|-------------------|------------------------------------------------|
|
||||
| `git_user.name` | agent's if non-empty, else bottle's |
|
||||
| `git_user.email` | agent's if non-empty, else bottle's |
|
||||
|
||||
This is the same per-field overlay `_merge_bottles` already applies
|
||||
for `extends:` (`manifest.py:1212`). Empty string = "not set", the
|
||||
same predicate the provisioner's `is_empty()` uses. All other bottle
|
||||
fields are returned unchanged.
|
||||
|
||||
### Where the overlay lives
|
||||
|
||||
```
|
||||
def bottle_for(self, agent_name) -> Bottle:
|
||||
agent = self.agents[agent_name]
|
||||
bottle = self.bottles[agent.bottle]
|
||||
if agent.git_user.is_empty():
|
||||
return bottle
|
||||
merged = GitUser(
|
||||
name=agent.git_user.name or bottle.git_user.name,
|
||||
email=agent.git_user.email or bottle.git_user.email,
|
||||
)
|
||||
return dataclasses.replace(bottle, git_user=merged)
|
||||
```
|
||||
|
||||
Both `provision/git.py` paths (docker + smolmachines) already call
|
||||
`bottle_for(agent_name)` and read `bottle.git_user`, so they pick up
|
||||
the merged identity with no edits. The dashboard / `info` / preflight
|
||||
surfaces that print bottle config go through the same resolution.
|
||||
|
||||
### Provenance display
|
||||
|
||||
A sibling `Manifest.git_identity_summary(agent_name) -> str | None`
|
||||
returns the effective identity annotated per field, e.g.:
|
||||
|
||||
```
|
||||
identity : name=claude-implementer (agent), email=eric+claude-implementer@dideric.is (bottle)
|
||||
```
|
||||
|
||||
`(agent)` when the agent's `git.user` supplied that field, `(bottle)`
|
||||
when it fell through to the referenced bottle. Returns `None` when no
|
||||
identity is set at either level (callers omit the line). Both
|
||||
`bottle_plan.py` preflights and `cli/info.py` print it; `info` today
|
||||
prints `git remotes` but not the identity, so this adds the line.
|
||||
|
||||
### `Agent` dataclass
|
||||
|
||||
`Agent` gains `git_user: GitUser = GitUser()`. `Agent.from_dict`
|
||||
parses the optional `git.user` and rejects non-`user` git keys.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **PRD (this commit).** Sets the design.
|
||||
2. **Schema + overlay + tests.**
|
||||
- Add `"git"` to `_AGENT_KEYS`.
|
||||
- Add `git_user: GitUser = GitUser()` to `Agent`; parse it in
|
||||
`Agent.from_dict`, reusing `GitUser.from_dict`; reject any
|
||||
`git` key other than `user` with a bottle-only message.
|
||||
- Thread `fm.get("git")` through `_load_agents_from_dir`'s
|
||||
`agent_dict`.
|
||||
- Apply the per-field overlay in `Manifest.bottle_for()`.
|
||||
- Add `Manifest.git_identity_summary()` and print it in both
|
||||
`bottle_plan.py` preflights and `cli/info.py`.
|
||||
- Unit tests: agent name+email overlay; agent name-only (email
|
||||
falls through to bottle); agent email-only; agent identity with
|
||||
a bottle that declares none; agent `git.remotes` dies; agent
|
||||
with no `git` block unchanged; bottle-only behavior preserved;
|
||||
provenance summary returns the right `(agent)`/`(bottle)` tags
|
||||
and `None` when unset.
|
||||
3. **Docs + example.** README manifest section: note `git.user` is
|
||||
allowed on agents and overlays the bottle; `git.remotes` stays
|
||||
bottle-only. Collapse the example — `examples/agents/implementer.md`
|
||||
carries its own `git.user` against the shared `dev` bottle,
|
||||
demonstrating per-agent identity without an identity-only bottle.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Unit (must):** the overlay matrix above, the parse-time reject
|
||||
for agent `git.remotes`, and confirmation that `bottle_for` leaves
|
||||
every non-`git_user` bottle field untouched.
|
||||
- **No integration changes needed:** provisioners consume the
|
||||
already-merged `Bottle` via `bottle_for`; existing docker /
|
||||
smolmachines git-provisioning tests cover the consumption path.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Should a `git.user`-declaring agent be loadable as a Claude Code
|
||||
subagent file too?** The CC-passthrough keys (`name`, `model`, …)
|
||||
let one file double as `~/.claude/agents/*.md`; `git` is not a CC
|
||||
key, so a file carrying it won't round-trip as a CC subagent. Not a
|
||||
blocker (bot-bottle ignores unknown CC keys; CC ignores `git`), but
|
||||
worth noting. Out of scope.
|
||||
@@ -1,170 +0,0 @@
|
||||
# PRD 0028: git-gate new-branch push scan scope
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-05-29
|
||||
- **Issue:** #106
|
||||
|
||||
## Summary
|
||||
|
||||
git-gate's pre-receive hook scans the **entire ancestry** of a *new*
|
||||
branch for secrets, so any pre-existing finding in repo history blocks
|
||||
every new-branch push. Scope the scan to the commits a push actually
|
||||
introduces (`$new --not --all`) so a push is gated on what *it* adds,
|
||||
not on what's already on the gate/upstream. Also harden the forward
|
||||
`ssh` against hangs. Net: new branches can be pushed through the gate
|
||||
again, with no loss of leak-detection coverage.
|
||||
|
||||
## Problem
|
||||
|
||||
In `git_gate_render_hook()` (`bot_bottle/git_gate.py`) the pre-receive
|
||||
hook chooses the gitleaks revision range per ref:
|
||||
|
||||
```sh
|
||||
if [ "$old" = "$zero" ]; then
|
||||
log_opts="$new" # new branch: `git log <new>` = the FULL ancestry
|
||||
else
|
||||
log_opts="$old..$new" # existing branch: just the pushed delta
|
||||
fi
|
||||
gitleaks git --log-opts="$log_opts" ... # exit 1 if ANY finding in range
|
||||
```
|
||||
|
||||
For a **new** ref there is no `old` to diff against, so the hook passes
|
||||
`$new`, which `git log` expands to every commit reachable from the new
|
||||
tip. This repo's history contains 11 deliberately secret-shaped strings
|
||||
(demo manifests, `docs/demo.tape`, and the pipelock/sandbox-escape
|
||||
integration tests that exist *to exercise* the DLP). gitleaks reports
|
||||
`438 commits scanned … leaks found: 11`, the hook `exit 1`s, and the
|
||||
push is rejected. Confirmed live against issue #106's bottle: the
|
||||
branch never lands in the bare repo and is never forwarded.
|
||||
|
||||
Consequence: **no new branch can ever be pushed through git-gate** as
|
||||
long as a single historical finding exists — which is permanent.
|
||||
|
||||
Two adjacent problems surfaced while diagnosing #106:
|
||||
|
||||
- The rejection is **invisible to the client** — over the `git://` +
|
||||
smolmachines forward it presented as a ~75s silent hang, not a
|
||||
`remote: git-gate: gitleaks rejected …` message.
|
||||
- The forward `ssh` lacks `BatchMode`/`ConnectTimeout`, so an
|
||||
unreachable upstream or a prompt would hang the hook indefinitely.
|
||||
(Not the cause of #106 — the forward itself works — but a latent
|
||||
hang risk.)
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- A new-branch push is scanned **only for the commits it introduces**
|
||||
(reachable from `$new`, not from any ref the gate already has).
|
||||
- A new branch that adds no new findings **pushes successfully**, even
|
||||
though historical fixtures still trip a full-history scan.
|
||||
- A new branch that *does* introduce a finding is **still rejected**.
|
||||
- No reduction in leak coverage for the commits a push actually brings
|
||||
to the upstream (see "Security analysis").
|
||||
- Forward `ssh` fails fast (`BatchMode=yes` + `ConnectTimeout`) instead
|
||||
of hanging on a prompt/unreachable upstream.
|
||||
- Existing git-gate unit + integration tests stay green; new tests lock
|
||||
the scoped-scan behaviour.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Scrubbing the historical fixture findings.** They're intentional
|
||||
test/demo inputs; scoping the scan resolves the practical problem
|
||||
without rewriting history.
|
||||
- **Relaxing the existing-branch path.** `$old..$new` already scans the
|
||||
delta; this PRD only fixes the new-ref branch (and optionally unifies
|
||||
on `--not --all`, see Open questions).
|
||||
- **The client-visibility fix is investigation-gated.** Surfacing the
|
||||
rejection over the `git://` + smolmachines path may need separate work
|
||||
(sideband relay); tracked here but may land as a follow-up rather than
|
||||
block the scan-scope fix.
|
||||
|
||||
## Design
|
||||
|
||||
### Scoped scan
|
||||
|
||||
Replace the new-ref range with one that excludes everything the gate
|
||||
already knows:
|
||||
|
||||
```sh
|
||||
if [ "$new" = "$zero" ]; then
|
||||
continue # deletion: nothing to scan (unchanged)
|
||||
elif [ "$old" = "$zero" ]; then
|
||||
log_opts="$new --not --all" # new branch: only commits new to the gate
|
||||
else
|
||||
log_opts="$old..$new" # existing branch: the pushed delta
|
||||
fi
|
||||
```
|
||||
|
||||
`git log $new --not --all` = commits reachable from the pushed tip but
|
||||
**not** reachable from any ref already in the gate's bare repo.
|
||||
|
||||
### Security analysis (why excluding "already-on-the-gate" is safe)
|
||||
|
||||
Commits enter the gate's bare repo by exactly two paths:
|
||||
|
||||
1. **mirror-fetch from the upstream** — the bare repo is
|
||||
`remote add --mirror=fetch origin`, and the access-hook fetches the
|
||||
upstream before every upload-pack; and
|
||||
2. **a push through the gate** — which is gitleaks-scanned before it is
|
||||
forwarded.
|
||||
|
||||
Therefore every commit reachable from a gate ref is *already on the
|
||||
upstream* or *was already scanned when pushed*. A commit excluded by
|
||||
`--not --all` cannot be a new secret arriving at the upstream via this
|
||||
push:
|
||||
|
||||
- if it's already upstream, re-scanning changes nothing — the content is
|
||||
already there and blocking this branch wouldn't remove it; and
|
||||
- if it arrived via an earlier push, it was already scanned.
|
||||
|
||||
The only commits that can carry a *new* secret upstream are the ones the
|
||||
push introduces — exactly the set `$new --not --all` scans. An agent
|
||||
cannot pre-seed a secret commit as "already known" to dodge the scan: it
|
||||
can't write refs into the bare repo except by pushing (which scans), and
|
||||
the mirror refs come only from the trusted upstream.
|
||||
|
||||
**Invariant this relies on:** the bare repo's refs are populated *only*
|
||||
by upstream mirror-fetch and gitleaks-gated pushes. That holds in the
|
||||
current design (nothing writes refs out-of-band); revisit if that
|
||||
changes.
|
||||
|
||||
### Forward ssh hardening
|
||||
|
||||
Add `-o BatchMode=yes -o ConnectTimeout=<n>` to the hook's `ssh_cmd` so
|
||||
a prompt or unreachable upstream fails fast with a clear error instead
|
||||
of hanging the receive-pack.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **PRD (this commit).**
|
||||
2. **Hook scan scope + ssh hardening.** Edit `git_gate_render_hook()`:
|
||||
the new-ref range → `$new --not --all`; add `BatchMode`/`ConnectTimeout`
|
||||
to `ssh_cmd`. Unit tests in `test_git_gate.py` asserting the rendered
|
||||
hook uses the scoped range for new refs and the hardened ssh flags.
|
||||
3. **Integration coverage.** A new-branch push carrying no new finding
|
||||
succeeds through a gate whose history contains a fixture finding; a
|
||||
new-branch push that introduces a finding is still rejected.
|
||||
4. **(Optional / follow-up) client visibility.** Make a gitleaks/forward
|
||||
rejection reach the client as a `remote:` error over the git:// +
|
||||
smolmachines path.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Unit (must):** rendered-hook assertions — new-ref uses
|
||||
`$new --not --all`, existing-ref still `$old..$new`, deletion still
|
||||
skipped; ssh_cmd carries `BatchMode=yes` + a `ConnectTimeout`.
|
||||
- **Integration (should):** against a real gate seeded with a
|
||||
fixture-bearing history, a clean new branch forwards to the upstream;
|
||||
a new branch with a planted secret is rejected. Skips cleanly on hosts
|
||||
that can't run the bundle (same shape as the existing git-gate
|
||||
integration test).
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Unify both branches on `--not --all`?** It's also more robust than
|
||||
`$old..$new` for non-fast-forward/force pushes (which can skip commits
|
||||
off the direct path). Tempting to use it for the existing-ref case
|
||||
too; deferred to keep this change tight, but worth a follow-up.
|
||||
- **Client visibility mechanism.** Whether the silent-hang is a git
|
||||
daemon sideband-relay issue or specific to the smolmachines forward
|
||||
needs a focused repro before committing to a fix.
|
||||
@@ -1,198 +0,0 @@
|
||||
# PRD 0029: Provider auth credentials through egress
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis-codex
|
||||
- **Created:** 2026-05-29
|
||||
- **Issue:** #109
|
||||
|
||||
## Summary
|
||||
|
||||
Allow provider bottles to inject host credentials into the egress
|
||||
sidecar without exposing them to the agent. Codex uses
|
||||
`agent_provider.forward_host_credentials` for ChatGPT/device-login
|
||||
access tokens. Claude uses `agent_provider.auth_token` to name the host
|
||||
env var holding its OAuth token, which egress injects on
|
||||
`api.anthropic.com` requests.
|
||||
|
||||
## Problem
|
||||
|
||||
Codex bottles can reach OpenAI hosts after they are added to egress, but
|
||||
requests to Codex's ChatGPT-backed API endpoints still fail with HTTP
|
||||
403 when the egress route is unauthenticated. The egress proxy strips
|
||||
agent-originated `Authorization` headers and only re-injects auth for
|
||||
routes that declare an egress-owned token. Bare `api.openai.com` or
|
||||
`chatgpt.com` routes therefore forward Codex requests without the
|
||||
ChatGPT bearer token.
|
||||
|
||||
Copying the host `~/.codex/auth.json` into the agent would solve auth
|
||||
mode detection but would also put access and refresh material inside the
|
||||
agent sandbox. That cuts against bot-bottle's credential minimization
|
||||
model: provider credentials should live in the sidecar boundary when
|
||||
possible, not in the agent.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- A Codex bottle with host ChatGPT auth can call Codex's
|
||||
`api.openai.com` and `chatgpt.com` endpoints through egress.
|
||||
- Host credential forwarding happens only when the bottle declares
|
||||
`agent_provider.forward_host_credentials: true`.
|
||||
- The agent container does not receive `OPENAI_API_KEY`,
|
||||
`CODEX_ACCESS_TOKEN`, or real `tokens.access_token` /
|
||||
`tokens.refresh_token` values.
|
||||
- The agent container receives only a dummy Codex `auth.json` that
|
||||
preserves the host auth-mode shape, keeps the selected ChatGPT
|
||||
account id, and replaces credential values with placeholders.
|
||||
- Egress route files remain non-secret: they contain only host/path/auth
|
||||
slot metadata, never token values.
|
||||
- Missing, API-key, malformed, or expired host Codex auth fails
|
||||
launch with a clear operator-facing message.
|
||||
- Existing Claude OAuth placeholder behavior remains unchanged.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Refreshing Codex tokens in the sidecar. The first cut reads the host's
|
||||
current access token at launch; operators can restart after host Codex
|
||||
refreshes auth.
|
||||
- Copying host `~/.codex/auth.json` credentials into the agent.
|
||||
- Allowing arbitrary host credential forwarding beyond the two providers
|
||||
covered here (Codex ChatGPT/device-login and Claude OAuth).
|
||||
- Hot-applying new authenticated Codex routes to an existing running
|
||||
sidecar. The current hot-apply path cannot safely populate new token
|
||||
env slots in an already-running container.
|
||||
|
||||
## Scope
|
||||
|
||||
### In scope
|
||||
|
||||
- Add `agent_provider.forward_host_credentials` to the bottle manifest
|
||||
schema, defaulting to `false`.
|
||||
- Support the flag for `agent_provider.template: codex`.
|
||||
- Add `agent_provider.auth_token` to the bottle manifest schema.
|
||||
- Support the field for `agent_provider.template: claude`: the named
|
||||
host env var is forwarded only into the egress sidecar as the Bearer
|
||||
token for `api.anthropic.com`, and a placeholder
|
||||
`CLAUDE_CODE_OAUTH_TOKEN` is set in the agent so the Claude Code CLI
|
||||
starts without a real credential.
|
||||
- Remove the `claude_code_oauth` egress route role, which previously
|
||||
required operators to declare the OAuth route manually. The provisioner
|
||||
now injects it from `auth_token`.
|
||||
- Read host Codex auth from `$CODEX_HOME/auth.json` when `CODEX_HOME` is
|
||||
set, otherwise from `~/.codex/auth.json`.
|
||||
- Extract only `tokens.access_token` for egress injection.
|
||||
- Generate a dummy agent-side `auth.json` from the host auth file's
|
||||
mode and key shape, without copying real token values.
|
||||
- Validate that host auth is not API-key mode and the access token is
|
||||
present, JWT-shaped, and not expired.
|
||||
- Add or upgrade `api.openai.com` and `chatgpt.com` egress routes to
|
||||
inject that access token via a shared `EGRESS_TOKEN_N` sidecar env
|
||||
slot.
|
||||
- Pass the extracted token only into the sidecar compose/run
|
||||
environment, alongside other egress token values.
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Sidecar-owned refresh using `tokens.refresh_token`.
|
||||
- Sharing full Codex auth state with the agent.
|
||||
- Supporting host credential forwarding for non-Codex providers.
|
||||
|
||||
## Design
|
||||
|
||||
### Manifest
|
||||
|
||||
Extend `agent_provider`:
|
||||
|
||||
```yaml
|
||||
agent_provider:
|
||||
template: codex
|
||||
forward_host_credentials: true
|
||||
```
|
||||
|
||||
The field defaults to `false`. If set on a non-Codex provider, manifest
|
||||
validation should reject it until that provider has a concrete,
|
||||
credential-minimizing implementation.
|
||||
|
||||
### Host auth extraction
|
||||
|
||||
At prepare/launch time, when the flag is enabled for Codex:
|
||||
|
||||
1. Resolve the host Codex home directory from `$CODEX_HOME`, falling
|
||||
back to `~/.codex`.
|
||||
2. Parse `auth.json`.
|
||||
3. Require user/device auth mode rather than API-key auth.
|
||||
4. Require a non-empty `tokens.access_token`.
|
||||
5. Parse the JWT payload enough to require an `exp` claim in the future.
|
||||
6. Return only the access token value to the launch path.
|
||||
|
||||
Errors should name the missing or invalid condition and point the
|
||||
operator at `codex login --device-auth`, without printing token values.
|
||||
|
||||
### Egress route
|
||||
|
||||
When forwarding host Codex credentials, the effective egress route table
|
||||
should contain authenticated `api.openai.com` and `chatgpt.com` routes.
|
||||
If the bottle already declares either host as a bare-pass route, upgrade
|
||||
it in the effective route table rather than requiring a duplicate
|
||||
manifest entry. If the bottle already declares an authenticated route for
|
||||
either host, fail rather than guessing whether to override
|
||||
operator-provided auth, unless that route already uses the synthetic
|
||||
Codex host credential token reference.
|
||||
|
||||
The rendered route should look like any other egress-owned auth route:
|
||||
|
||||
```yaml
|
||||
routes:
|
||||
- host: "api.openai.com"
|
||||
auth_scheme: "Bearer"
|
||||
token_env: "EGRESS_TOKEN_N"
|
||||
- host: "chatgpt.com"
|
||||
auth_scheme: "Bearer"
|
||||
token_env: "EGRESS_TOKEN_N"
|
||||
```
|
||||
|
||||
The access token value is supplied through the sidecar process
|
||||
environment for that `EGRESS_TOKEN_N` slot. It must not be written to
|
||||
`routes.yaml`, compose files, env files, logs, or user-facing output.
|
||||
|
||||
### Data flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
H["Host ~/.codex/auth.json"] --> L["bot-bottle launch"]
|
||||
L -->|access token only| S["egress sidecar env"]
|
||||
L -->|dummy auth.json only| A
|
||||
A["Codex agent"] -->|HTTPS via proxy, auth stripped| E["egress"]
|
||||
E -->|Bearer injected from env| C["api.openai.com / chatgpt.com"]
|
||||
```
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **PRD first.** Land this document as the first commit on the feature
|
||||
branch.
|
||||
2. **Manifest schema.** Add `forward_host_credentials`, validation, and
|
||||
unit tests.
|
||||
3. **Host Codex auth reader.** Add a small stdlib-only helper for
|
||||
parsing and validating host Codex auth without printing values.
|
||||
4. **Effective egress route.** Add/upgrade the Codex API routes when the
|
||||
flag is enabled, and add tests for bare route upgrade,
|
||||
missing-route insertion, and authenticated-route conflict.
|
||||
5. **Agent auth marker.** Provision a dummy Codex `auth.json` into the
|
||||
agent home so Codex selects the host's user/device auth branch while
|
||||
real credentials stay in egress.
|
||||
6. **Launch wiring.** Pass the host access token into the egress sidecar
|
||||
env for Docker and smolmachines without exposing it to the agent.
|
||||
7. **Docs and tests.** Update README examples and run the unit suite.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a later version support sidecar refresh using the host refresh
|
||||
token, or should restart-on-expiry remain the policy?
|
||||
- Should telemetry hosts such as `ab.chatgpt.com` stay blocked by
|
||||
default even when Codex ChatGPT auth is enabled?
|
||||
|
||||
## References
|
||||
|
||||
- Gitea issue #109: Codex ChatGPT auth should inject host access token
|
||||
via egress.
|
||||
- PRD 0017: Egress-proxy — universal MITM with path filtering + auth
|
||||
injection.
|
||||
- PRD 0026: Agent provider templates.
|
||||
@@ -1,121 +0,0 @@
|
||||
# PRD 0030: Deduplicate egress token resolution across backends
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-06-02
|
||||
- **Issue:** #118
|
||||
|
||||
## Summary
|
||||
|
||||
Eliminate the duplicated egress token resolution block — which resolved
|
||||
manifest-declared tokens and the Codex host credential — by moving
|
||||
provider-specific token reading into `AgentProvisionPlan.provisioned_env`
|
||||
at prepare time, and having both backends merge that map into `os.environ`
|
||||
before calling the now-generic `egress_resolve_token_values`.
|
||||
|
||||
## Problem
|
||||
|
||||
The same logic block appeared in two places:
|
||||
|
||||
- `bot_bottle/backend/docker/launch.py` (~line 183): inline inside
|
||||
`launch()`, building `token_values` before `compose_up`.
|
||||
- `bot_bottle/backend/smolmachines/launch.py` (~line 422): the private
|
||||
`_resolve_token_env` helper, called before `_bundle.start_bundle`.
|
||||
|
||||
Both blocks:
|
||||
1. Short-circuit to `{}` when there are no egress routes.
|
||||
2. Call `egress_resolve_token_values(token_env_map, host_env)` to
|
||||
resolve ordinary manifest-declared token refs.
|
||||
3. Check `agent_provider.forward_host_credentials` and, when true,
|
||||
call `codex_host_access_token` and slot the result into any
|
||||
`token_env` whose `token_ref` is `CODEX_HOST_CREDENTIAL_TOKEN_REF`.
|
||||
|
||||
The duplication means any change to step 3 must be applied twice.
|
||||
PRD 0029, which introduced `forward_host_credentials`, already had to
|
||||
wire both backends; the next change would too. This is a near-certain
|
||||
future sync bug.
|
||||
|
||||
`egress_resolve_token_values` also carried a sentinel `continue` skip
|
||||
for `CODEX_HOST_CREDENTIAL_TOKEN_REF`, which tied an otherwise generic
|
||||
egress helper to a Codex-specific contract.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- The `forward_host_credentials` resolution logic exists in exactly one
|
||||
place in the codebase.
|
||||
- Both `docker/launch.py` and `smolmachines/launch.py` call
|
||||
`egress_resolve_token_values` with no provider-specific arguments.
|
||||
- `egress_resolve_token_values` is fully generic — it neither knows nor
|
||||
cares about provider identity or the `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
||||
sentinel.
|
||||
- No behaviour change for either backend.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changes to token resolution semantics. This is a pure refactor.
|
||||
- Adding support for any new credential type or provider.
|
||||
- Consolidating any other backend differences beyond this one block.
|
||||
|
||||
## Design
|
||||
|
||||
### `AgentProvisionPlan.provisioned_env`
|
||||
|
||||
Add `provisioned_env: dict[str, str]` (default empty) to
|
||||
`AgentProvisionPlan`. This map holds host-side secrets that the
|
||||
provisioning stage resolved and that egress needs injected into the
|
||||
sidecar environ.
|
||||
|
||||
When `forward_host_credentials=True` for Codex, `agent_provision_plan`
|
||||
calls `codex_host_access_token(host_env)` and stores the result under
|
||||
`CODEX_HOST_CREDENTIAL_TOKEN_REF`. This is already the prepare-time
|
||||
stage where `write_codex_dummy_auth_file` runs, so the access-token
|
||||
read is colocated with all other Codex-specific provisioning.
|
||||
|
||||
### Backend call sites
|
||||
|
||||
Both backends merge `provisioned_env` over `os.environ` before calling
|
||||
`egress_resolve_token_values`:
|
||||
|
||||
```python
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||
```
|
||||
|
||||
`provisioned_env` values take precedence over same-named host env vars,
|
||||
so the Codex token slot resolves from the map written at prepare time
|
||||
rather than from a raw `os.environ` lookup.
|
||||
|
||||
### `egress_resolve_token_values`
|
||||
|
||||
Remove the `CODEX_HOST_CREDENTIAL_TOKEN_REF` sentinel `continue` skip.
|
||||
The function now resolves every slot in `token_env_map` from `host_env`
|
||||
without special-casing any key name. The `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
||||
key is present in `effective_env` (injected by `provisioned_env`) exactly
|
||||
when it is needed.
|
||||
|
||||
## Implementation
|
||||
|
||||
- `bot_bottle/agent_provider.py`: add `provisioned_env` field; populate
|
||||
it for Codex when `forward_host_credentials=True`.
|
||||
- `bot_bottle/egress.py`: remove sentinel skip; remove
|
||||
`egress_resolve_token_values_with_provider` (the intermediate function
|
||||
that was introduced and then superseded by this design); drop the
|
||||
`codex_auth` import.
|
||||
- `bot_bottle/backend/docker/launch.py`: replace the provider-aware
|
||||
resolution block with the `effective_env` merge.
|
||||
- `bot_bottle/backend/smolmachines/launch.py`: same replacement in
|
||||
`_resolve_token_env`.
|
||||
- `tests/unit/test_agent_provider.py`: add tests verifying
|
||||
`provisioned_env` is populated (or empty) for Codex with and without
|
||||
`forward_host_credentials`.
|
||||
- `tests/unit/test_egress.py`: remove `TestResolveTokenValuesWithProvider`;
|
||||
replace the sentinel-skip test with a test verifying the Codex token
|
||||
ref resolves normally when present in `host_env`.
|
||||
|
||||
## References
|
||||
|
||||
- Issue #118: Deduplicate egress token resolution across backends.
|
||||
- Issue #117: Complexity hotspots in launch, egress, and auth paths
|
||||
(source of the finding).
|
||||
- PRD 0029: Provider auth credentials through egress (introduced the
|
||||
duplicated block).
|
||||
@@ -1,241 +0,0 @@
|
||||
# PRD 0031: Simplify egress route merge and consolidate Route types
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-06-02
|
||||
- **Issue:** #120
|
||||
|
||||
## Summary
|
||||
|
||||
Replace `_merge_provider_route`'s five-case nested conditional with a
|
||||
flat provisioned-wins merge, and make the mapping between the host-side
|
||||
`EgressRoute` and the addon's `Route` explicit in one place. Covers the
|
||||
two remaining open tasks from the #117 hotspot review.
|
||||
|
||||
## Problem
|
||||
|
||||
### 1. `_merge_provider_route` branching
|
||||
|
||||
`_merge_provider_route` in `bot_bottle/egress.py` handles five distinct
|
||||
cases in a single function with interleaved conditions:
|
||||
|
||||
1. **append-new** — host not in manifest; append a fresh route.
|
||||
2. **upgrade-bare** — host found, no existing auth; adopt provider auth.
|
||||
3. **no-op** — host found, same auth; return unchanged.
|
||||
4. **tls-passthrough upgrade** — same as no-op but provider sets
|
||||
`tls_passthrough=True`; flip the flag on the existing route.
|
||||
5. **conflict-die** — host found, different auth; hard error.
|
||||
|
||||
Cases 3 and 4 share a block with no-op as the invisible fall-through.
|
||||
`_find_or_alloc_token_env` is duplicated between cases 2 and 1. In-place
|
||||
replacements spell out every `EgressRoute` field explicitly, so a new
|
||||
field added to the dataclass silently drops its value in any replacement
|
||||
site that wasn't updated.
|
||||
|
||||
The root cause of the complexity is that the current merge tries to be
|
||||
cooperative: it lets manifest routes coexist with provider routes and
|
||||
attempts to upgrade bare manifest entries. This makes sense if the
|
||||
manifest is authoritative, but the actual intended hierarchy is the
|
||||
opposite — provider routes claim their hosts outright and the manifest
|
||||
fills in what's left.
|
||||
|
||||
### 2. Three-way Route type fragmentation
|
||||
|
||||
`EgressRoute` (in `egress.py`) and `egress_addon_core.Route` are
|
||||
separate dataclasses with overlapping but not identical field sets:
|
||||
|
||||
| Field | `EgressRoute` | addon `Route` |
|
||||
|---|---|---|
|
||||
| `host` | ✓ | ✓ |
|
||||
| `path_allowlist` | ✓ | ✓ |
|
||||
| `auth_scheme` | ✓ | ✓ |
|
||||
| `token_env` | ✓ | ✓ |
|
||||
| `token_ref` | ✓ (host-side) | — |
|
||||
| `roles` | ✓ (host-side) | — |
|
||||
| `tls_passthrough` | ✓ (pipelock concern) | — |
|
||||
|
||||
`egress_render_routes` serialises `EgressRoute` fields to YAML; the
|
||||
addon's `load_routes` deserialises that YAML into `Route` objects. If a
|
||||
field is added to `EgressRoute` that should appear in the YAML, both
|
||||
`egress_render_routes` and `_parse_one` must be updated consistently.
|
||||
The render function spells the field list out inline with no reference
|
||||
to the addon's parser, so divergence is silent until runtime.
|
||||
|
||||
`egress_addon_core.Route` cannot be replaced by `EgressRoute` — the
|
||||
addon file is copied flat into the sidecar container image (`/app/`) and
|
||||
has no access to the `bot_bottle` package. The types must stay separate;
|
||||
the risk is that they drift.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `egress_routes_for_bottle` implements a flat provisioned-wins merge:
|
||||
provider routes claim their hosts; manifest routes for unclaimed hosts
|
||||
append. No upgrade logic, no conflict detection.
|
||||
- Token slot assignment is a single pass over the merged list, not
|
||||
interleaved with the merge.
|
||||
- `egress_render_routes` uses a single `_route_to_yaml_fields` helper
|
||||
that explicitly lists the addon-visible fields, creating one place
|
||||
where the `EgressRoute`→`Route` mapping is spelled out.
|
||||
- All existing `TestProviderRouteMerge` and `TestRenderRoutes` tests
|
||||
pass (adjusting assertions for any semantics changes described below).
|
||||
- No behaviour change for existing manifests that don't trigger the
|
||||
conflict-die or upgrade-bare paths.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Merging `EgressRoute` and `egress_addon_core.Route` into one class
|
||||
(impossible: addon runs in a stdlib-only container environment).
|
||||
- Changing what the addon does with a route once it has one.
|
||||
- Changing `decide()` or `is_git_push_request()` in `egress_addon_core`
|
||||
— those are already pure functions with good separation.
|
||||
|
||||
## Design
|
||||
|
||||
### Merge: provisioned wins
|
||||
|
||||
The new hierarchy: **provisioned routes own their hosts; manifest routes
|
||||
fill the gaps.**
|
||||
|
||||
```
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
|
||||
effective = list(provider_routes)
|
||||
effective += [r for r in manifest_routes if r.host.lower() not in provisioned_hosts]
|
||||
```
|
||||
|
||||
Token slot assignment runs as a final pass over `effective` in order:
|
||||
|
||||
```python
|
||||
def _assign_token_slots(
|
||||
routes: list[EgressRoute],
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
slot_for_ref: dict[str, str] = {}
|
||||
out: list[EgressRoute] = []
|
||||
for r in routes:
|
||||
if r.auth_scheme and r.token_ref and not r.token_env:
|
||||
token_env = slot_for_ref.get(r.token_ref)
|
||||
if token_env is None:
|
||||
token_env = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
||||
slot_for_ref[r.token_ref] = token_env
|
||||
r = dataclasses.replace(r, token_env=token_env)
|
||||
out.append(r)
|
||||
return tuple(out)
|
||||
```
|
||||
|
||||
This replaces `_merge_provider_route`, `_find_or_alloc_token_env`, and
|
||||
the slot-assignment loop inside `egress_manifest_routes`.
|
||||
|
||||
#### Semantics change
|
||||
|
||||
Under the old design, a manifest route for a provisioned host with a
|
||||
*different* `auth_scheme` or `token_ref` raised a hard error. Under
|
||||
provisioned-wins, the manifest entry is silently dropped. Operators who
|
||||
relied on the conflict error to catch misconfigurations should audit
|
||||
their manifests, but in practice this path was only reachable when a
|
||||
manifest declared auth for `api.openai.com` or `chatgpt.com` with a
|
||||
token ref other than `CODEX_HOST_CREDENTIAL_TOKEN_REF` while also
|
||||
enabling `forward_host_credentials` — an unlikely combination.
|
||||
|
||||
Similarly, the "upgrade-bare" path (provider adopts a bare manifest
|
||||
route's `path_allowlist`) is dropped: a provisioned host takes the
|
||||
provider route's fields wholesale, and the manifest's `path_allowlist`
|
||||
for that host is ignored.
|
||||
|
||||
### Route type mapping: `_route_to_yaml_fields`
|
||||
|
||||
Add a pure function in `egress.py`:
|
||||
|
||||
```python
|
||||
def _route_to_yaml_fields(r: EgressRoute) -> dict:
|
||||
"""Return the addon-visible fields for one route.
|
||||
|
||||
This is the single authoritative mapping between `EgressRoute`
|
||||
(host-side) and `egress_addon_core.Route` (sidecar-side). If a
|
||||
field is added to `Route` that must appear in the YAML, add it
|
||||
here and in `egress_addon_core._parse_one` together."""
|
||||
fields: dict = {"host": r.host}
|
||||
if r.auth_scheme and r.token_env:
|
||||
fields["auth_scheme"] = r.auth_scheme
|
||||
fields["token_env"] = r.token_env
|
||||
if r.path_allowlist:
|
||||
fields["path_allowlist"] = list(r.path_allowlist)
|
||||
return fields
|
||||
```
|
||||
|
||||
`egress_render_routes` delegates to it:
|
||||
|
||||
```python
|
||||
for r in routes:
|
||||
f = _route_to_yaml_fields(r)
|
||||
lines.append(f' - host: "{f["host"]}"')
|
||||
if "auth_scheme" in f:
|
||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||
lines.append(f' token_env: "{f["token_env"]}"')
|
||||
if "path_allowlist" in f:
|
||||
lines.append(" path_allowlist:")
|
||||
for p in f["path_allowlist"]:
|
||||
lines.append(f' - "{p}"')
|
||||
```
|
||||
|
||||
The docstring on `_route_to_yaml_fields` is the explicit callout to
|
||||
update both it and `_parse_one` together when the schema changes.
|
||||
|
||||
### `egress_manifest_routes` / `egress_routes_for_bottle`
|
||||
|
||||
`egress_manifest_routes` becomes a pure lifter with no slot assignment:
|
||||
it reads each manifest route entry and returns an `EgressRoute` with
|
||||
`token_env=""` (the slot to be filled later). The function's docstring
|
||||
currently promises slot assignment; that promise moves to
|
||||
`egress_routes_for_bottle`.
|
||||
|
||||
`egress_routes_for_bottle` becomes:
|
||||
|
||||
```python
|
||||
def egress_routes_for_bottle(
|
||||
bottle: Bottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
merged = list(provider_routes) + [
|
||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||
]
|
||||
return _assign_token_slots(merged)
|
||||
```
|
||||
|
||||
### Mitmproxy request logic
|
||||
|
||||
`egress_addon_core.decide()` and `is_git_push_request()` are already
|
||||
pure functions; `egress_addon.EgressAddon.request()` is the minimal
|
||||
mitmproxy glue (read host/path/headers from flow → call pure functions
|
||||
→ apply result to flow). This split is already clean and requires no
|
||||
structural change in this PRD.
|
||||
|
||||
## Test impact
|
||||
|
||||
- **`TestProviderRouteMerge`**: the `test_provider_route_upgrades_bare_manifest_route`
|
||||
test asserts that a provider route preserves a bare manifest route's
|
||||
`path_allowlist`. Under provisioned-wins that `path_allowlist` is
|
||||
dropped. Update the test to reflect the new semantics.
|
||||
- **`test_provider_route_conflicts_with_different_authed_manifest_route`**:
|
||||
the conflict-die case no longer exists. Remove this test.
|
||||
- All other merge and render tests should pass without modification.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **PRD (this commit).** Sets the design.
|
||||
2. **Merge refactor.** Replace `_merge_provider_route` and
|
||||
`_find_or_alloc_token_env` with `_assign_token_slots` and the
|
||||
flat provisioned-wins logic in `egress_routes_for_bottle`. Strip
|
||||
slot assignment from `egress_manifest_routes`.
|
||||
3. **Render consolidation.** Add `_route_to_yaml_fields`; update
|
||||
`egress_render_routes` to use it.
|
||||
4. **Test updates.** Adjust `TestProviderRouteMerge` for the semantics
|
||||
changes above; confirm all render tests pass.
|
||||
|
||||
## References
|
||||
|
||||
- Issue #120: Refactor `_merge_provider_route` (expanded to include
|
||||
Route type fragmentation).
|
||||
- Issue #117: Complexity hotspots — source of both findings.
|
||||
- PRD 0030: Deduplicate egress token resolution (prior egress cleanup).
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user