Compare commits
130 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| cc0c952d0b | |||
| 8c9d4fbc46 | |||
| b9ab1263c2 | |||
| 9282bceaf8 | |||
| 3e50079bcc | |||
| cf9aaf68e7 | |||
| 4cf2cfc55d | |||
| 7c285fde7a | |||
| 64ac204c05 | |||
| 59fd132b9d | |||
| f427d35e72 | |||
| 1105d9a269 | |||
| 46e596d0b1 | |||
| a3a8a01b09 | |||
| 941f316462 | |||
| be3defe5d8 | |||
| 3885e2f5ad | |||
| a08829573d | |||
| d5fcbe53ef | |||
| 6150497b47 | |||
| 5308d53288 | |||
| d01f4b6613 | |||
| 44273be9eb | |||
| 096c7b8196 | |||
| 0432a5d3ff | |||
| fcd1b34e49 | |||
| a0762ac2d3 | |||
| 53219a55e1 | |||
| 71ac555f25 | |||
| f25fa589fe | |||
| 4fdf354b4f | |||
| 5a2011c48f | |||
| 19ebcd52a1 | |||
| 2c061d9cd9 | |||
| cceb300d58 | |||
| b63927368a | |||
| 4319b4ef3b | |||
| 71005d56e2 | |||
| 96b0c3f1fa | |||
| 3087a9aa8b | |||
| e43f75dd1b | |||
| 4ad1ff3898 | |||
| a3d9ac9605 | |||
| 70c9f7254c | |||
| b9108339e7 | |||
| e5b5dd16f1 | |||
| cf76d1a245 | |||
| 717a9126e1 | |||
| 8830306101 | |||
| 1c242b0ad9 | |||
| f95ef0c446 | |||
| 6e954da9b7 | |||
| 9185c145a1 | |||
| a79ef61b62 | |||
| 0a8bba58c7 | |||
| 2247d730cd | |||
| 3472e06efb | |||
| 82ce5d3034 | |||
| 7c260eeff9 | |||
| fe6059e4a6 | |||
| 31708abfad | |||
| 1b34b1df85 | |||
| 51831bf9c0 | |||
| 8f28bd81a7 | |||
| 662e3e1f95 | |||
| 6315456a59 | |||
| a81f0ffa49 | |||
| c39bbe265b | |||
| 0d922371b0 | |||
| fe97b6014d | |||
| 07c8593999 | |||
| f15721b424 | |||
| 10d0872043 | |||
| ae33d1abfb | |||
| f596464f3f | |||
| e528d5c5af | |||
| 0e29bcc829 | |||
| 8c2b59ca94 | |||
| 75f0f9d907 | |||
| 6682357fbb | |||
| 2dd8113f7c | |||
| 36e3443d2e | |||
| d6ebd0d2eb | |||
| eb6bace84f | |||
| f8fc29ce87 | |||
| 938a0e05d6 | |||
| f768d3a853 | |||
| f32b7eb299 | |||
| de9bd7eb83 | |||
| 952dcd7eec | |||
| 59df0b0f0f | |||
| c0219dddd5 | |||
| 884cedc160 | |||
| 76a7921ae6 | |||
| c8ab0c67a8 | |||
| e808e81b87 | |||
| 36ce7aed4f | |||
| a5d83bdcdc | |||
| 8e6583fcb7 | |||
| ac1aa197d4 | |||
| 68e5097534 | |||
| f8a4e6f40b | |||
| a6332b9535 | |||
| 62dd7b2aa5 | |||
| 711cb9c194 | |||
| 0b80ffb16a | |||
| 2350cd11e0 | |||
| 6ea19a8d53 | |||
| 630e65e9a4 | |||
| 7bffaa791c | |||
| de2267d1b4 | |||
| dcaee53cec | |||
| cea832b21d | |||
| 50baf63669 | |||
| 6c673bece6 | |||
| 9dc0dfd5ee | |||
| 2ea73e40a8 | |||
| 7b2474a5d3 | |||
| 847baa84be | |||
| 99ec267c74 | |||
| 848515e5d4 | |||
| ae1531835d | |||
| 5c5f576df0 | |||
| d329e511fd | |||
| 1308e61c7e | |||
| 2141a85884 | |||
| ccbed97776 | |||
| 1df78ee77f | |||
| c840182d12 | |||
| 7b4c1cd091 |
@@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
name: quality-eval
|
||||||
|
description: Use when the user asks to objectively evaluate, score, rate, audit, or quality-gate code, codebases, files, pull requests, or snippets using a strict 5-dimension engineering rubric with scores and refactoring steps.
|
||||||
|
metadata:
|
||||||
|
short-description: Score code quality with a strict rubric
|
||||||
|
---
|
||||||
|
|
||||||
|
# Quality Eval
|
||||||
|
|
||||||
|
## Role
|
||||||
|
|
||||||
|
Act as a Staff Software Engineer and automated quality gate. Evaluate code objectively against the rubric below, surface hidden anti-patterns, and provide a mathematical grade with atomic refactoring steps.
|
||||||
|
|
||||||
|
## Evaluation Rules
|
||||||
|
|
||||||
|
- Evaluate only against the five rubric dimensions.
|
||||||
|
- Be candid. Do not inflate scores for politeness.
|
||||||
|
- Avoid generic advice. Every recommendation must name a specific code location, behavior, or pattern and include a concrete improvement direction.
|
||||||
|
- Inspect the code before scoring. For codebases, read enough representative files, tests, and architecture boundaries to justify the scope.
|
||||||
|
- When exact line numbers are available, cite them.
|
||||||
|
- Do not reveal private chain-of-thought. In the required `Chain of Thought Analysis` section, provide a concise, step-by-step audit rationale with observable findings and score justifications.
|
||||||
|
|
||||||
|
## Rubric
|
||||||
|
|
||||||
|
Score each dimension from 1 to 5 using these anchors:
|
||||||
|
|
||||||
|
| Dimension | Score 1 (Fail) | Score 3 (Pass) | Score 5 (Exemplary) |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| **Architecture** | Spaghettified; tight coupling; violated separation of concerns. | Modular but relies on leaky abstractions or mixed domains. | Strict domain isolation; follows SOLID; clear dependency inversion. |
|
||||||
|
| **Readability** | Cryptic naming; deep nesting (>3 levels); widespread DRY violations. | Idiomatic but features over-complex functions or sparse documentation. | Self-documenting; expressive naming; high cohesion; flat structure. |
|
||||||
|
| **Resilience** | Swallows errors blindly; lacks contextual logging; fragile to bad input. | Basic try/catch blocks present but lacks granular, typed error handling. | Explicit error boundaries; contextual logging; structured failure modes. |
|
||||||
|
| **Testability** | Hardcoded dependencies make mocking or isolated testing impossible. | Pure functions are testable, but side-effect heavy logic lacks test hooks. | Decoupled IO; deterministic execution; structured for unit and integration tests. |
|
||||||
|
| **SecOps** | Hardcoded secrets; O(n^2) bottlenecks; zero input sanitization. | Safe from obvious flaws but lacks deep defensive optimization. | Validated inputs; optimized algorithmic complexity; zero security debt. |
|
||||||
|
|
||||||
|
## Scoring Method
|
||||||
|
|
||||||
|
1. Determine the evaluated scope and primary language.
|
||||||
|
2. Identify concrete evidence for each dimension.
|
||||||
|
3. Assign integer dimension scores from 1 to 5.
|
||||||
|
4. Compute `composite_score` as the arithmetic mean of the five dimension scores, rounded to one decimal place.
|
||||||
|
5. Include code snippets only when they make a refactoring step more actionable.
|
||||||
|
|
||||||
|
## Required Output
|
||||||
|
|
||||||
|
Structure every response into exactly these three Markdown sections:
|
||||||
|
|
||||||
|
### 1. Chain of Thought Analysis
|
||||||
|
|
||||||
|
Provide a concise step-by-step audit rationale. Name specific files, functions, patterns, anti-patterns, and rubric anchors. Keep it evidence-based and do not include hidden private reasoning.
|
||||||
|
|
||||||
|
### 2. Normalized Score Report
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"evaluation_metadata": {
|
||||||
|
"target_scope": "string",
|
||||||
|
"primary_language": "string"
|
||||||
|
},
|
||||||
|
"metrics": {
|
||||||
|
"architecture_and_modularity": 0,
|
||||||
|
"readability_and_maintainability": 0,
|
||||||
|
"error_handling_and_resilience": 0,
|
||||||
|
"testability_and_mocking": 0,
|
||||||
|
"security_and_performance": 0
|
||||||
|
},
|
||||||
|
"composite_score": 0.0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Atomic Refactoring Playbook
|
||||||
|
|
||||||
|
* **High Priority (To lift Score 1/2 to 3):**
|
||||||
|
- [ ] Actionable, specific refactoring step with file/line/context reference.
|
||||||
|
* **Medium Priority (To lift Score 3 to 4/5):**
|
||||||
|
- [ ] Optimization or architectural pattern implementation step.
|
||||||
|
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
display_name: Quality Eval
|
||||||
|
short_description: Scores code quality with a strict five-dimension rubric and refactoring playbook.
|
||||||
|
default_prompt: Evaluate this code objectively using the quality-eval rubric and return the three-section score report.
|
||||||
@@ -28,14 +28,28 @@ the container lifecycle and the copying of skills and env vars into it.
|
|||||||
- `bot-bottle.json` — legacy manifest of named agents (env / skills / prompt
|
- `bot-bottle.json` — legacy manifest of named agents (env / skills / prompt
|
||||||
per agent), consumed by `cli.py`. See "Manifest" under
|
per agent), consumed by `cli.py`. See "Manifest" under
|
||||||
"Intended design".
|
"Intended design".
|
||||||
- `docs/INDEX.md` — pointer to the research notes.
|
- `docs/README.md` — docs overview; when to write which document.
|
||||||
- `docs/prds/` — product requirement docs.
|
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
|
||||||
- `docs/research/` — research notes (empty for now, kept tracked via `.gitkeep`).
|
- `docs/research/` — research notes (see `docs/research/README.md`).
|
||||||
|
- `docs/decisions/` — decision records (ADR-lite).
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Product requirement docs live in `docs/prds/`.
|
- Three kinds of doc, each with its own conventions in-folder; see
|
||||||
- Research notes live in `docs/research/`.
|
`docs/README.md` for when to write which:
|
||||||
|
- **PRDs** (`docs/prds/`) — one feature per file, numbered
|
||||||
|
`NNNN-kebab.md`. A `Status:` line tracks lifecycle: Draft → Active
|
||||||
|
(shipped to `main`) → Superseded/Retargeted. Format in
|
||||||
|
`docs/prds/README.md`.
|
||||||
|
- **Research notes** (`docs/research/`) — opinionated investigations;
|
||||||
|
unnumbered kebab-case, freeform and verdict-first. See
|
||||||
|
`docs/research/README.md`.
|
||||||
|
- **Decision records** (`docs/decisions/`) — ADR-lite, numbered
|
||||||
|
`NNNN-kebab.md`, for policies and non-feature decisions. See
|
||||||
|
`docs/decisions/README.md`.
|
||||||
|
- Keep decision rationale self-contained in the repo, not in Gitea
|
||||||
|
issue threads. Issues are an ephemeral inbox; the durable "why" lives
|
||||||
|
in a PRD, research note, or decision record.
|
||||||
- Low dependencies by default. The project is Python, stdlib-first (no
|
- Low dependencies by default. The project is Python, stdlib-first (no
|
||||||
runtime pip dependencies in the package itself; the only language
|
runtime pip dependencies in the package itself; the only language
|
||||||
runtime is the Python 3.13 used by the CLI + sidecars). Ask before
|
runtime is the Python 3.13 used by the CLI + sidecars). Ask before
|
||||||
|
|||||||
+1
-1
@@ -23,7 +23,7 @@ FROM node:22-slim
|
|||||||
# tool (curl itself, plus anything that shells out to it) works
|
# tool (curl itself, plus anything that shells out to it) works
|
||||||
# against pipelock's bumped TLS without the agent needing local DNS.
|
# against pipelock's bumped TLS without the agent needing local DNS.
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Install claude-code globally. Pinned to the version verified in the v1
|
# Install claude-code globally. Pinned to the version verified in the v1
|
||||||
|
|||||||
+2
-2
@@ -6,10 +6,10 @@
|
|||||||
FROM node:22-slim
|
FROM node:22-slim
|
||||||
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.134.0 \
|
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
||||||
&& npm cache clean --force
|
&& npm cache clean --force
|
||||||
|
|
||||||
USER node
|
USER node
|
||||||
|
|||||||
+3
-1
@@ -31,6 +31,7 @@
|
|||||||
# 9099 egress (mitmproxy, pipelock's upstream — not externally
|
# 9099 egress (mitmproxy, pipelock's upstream — not externally
|
||||||
# addressed by the agent)
|
# addressed by the agent)
|
||||||
# 9418 git-gate (git-daemon)
|
# 9418 git-gate (git-daemon)
|
||||||
|
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||||
# 9100 supervise (MCP HTTP)
|
# 9100 supervise (MCP HTTP)
|
||||||
|
|
||||||
# Stage 1: pipelock binary. The upstream pipelock image is a
|
# Stage 1: pipelock binary. The upstream pipelock image is a
|
||||||
@@ -81,6 +82,7 @@ COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
|||||||
COPY bot_bottle/supervise.py /app/supervise.py
|
COPY bot_bottle/supervise.py /app/supervise.py
|
||||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||||
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
||||||
|
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
|
||||||
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
|
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
|
||||||
RUN chmod +x /app/egress-entrypoint.sh
|
RUN chmod +x /app/egress-entrypoint.sh
|
||||||
|
|
||||||
@@ -97,7 +99,7 @@ RUN mkdir -p \
|
|||||||
|
|
||||||
# Documentation only — the compose renderer publishes whichever
|
# Documentation only — the compose renderer publishes whichever
|
||||||
# subset the bottle uses.
|
# subset the bottle uses.
|
||||||
EXPOSE 8888 9099 9418 9100
|
EXPOSE 8888 9099 9418 9420 9100
|
||||||
|
|
||||||
# WORKDIR matches Dockerfile.supervise's prior layout so the
|
# WORKDIR matches Dockerfile.supervise's prior layout so the
|
||||||
# in-app same-dir import in supervise_server.py stays deterministic.
|
# in-app same-dir import in supervise_server.py stays deterministic.
|
||||||
|
|||||||
@@ -157,14 +157,8 @@ and MCP endpoints resolve without an agent-side change.
|
|||||||
upstream has *now* (fail-closed if unreachable). The agent's
|
upstream has *now* (fail-closed if unreachable). The agent's
|
||||||
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
||||||
so push, fetch, clone, and pull all route through. The agent
|
so push, fetch, clone, and pull all route through. The agent
|
||||||
never sees the upstream credential. If the upstream's hostname
|
never sees the upstream credential. Brought up only when
|
||||||
isn't resolvable from the gate container (e.g. a Tailscale-only
|
`bottle.git` has entries. Design in `docs/prds/0008-git-gate.md`.
|
||||||
host whose public DNS points elsewhere), pin its IP via
|
|
||||||
`ExtraHosts: { "<hostname>": "<ip>" }` on the `bottle.git` entry —
|
|
||||||
the gate's `/etc/hosts` gets the override while the agent's
|
|
||||||
`insteadOf` rewrite still keys off the original hostname. Brought
|
|
||||||
up only when `bottle.git` has entries. Design in
|
|
||||||
`docs/prds/0008-git-gate.md`.
|
|
||||||
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
||||||
base, stdlib-only) that holds API tokens declared in
|
base, stdlib-only) that holds API tokens declared in
|
||||||
`bottle.cred_proxy.routes`. Each route names a `path`,
|
`bottle.cred_proxy.routes`. Each route names a `path`,
|
||||||
@@ -349,11 +343,40 @@ The `gitea-dev` bottle. Backs my work on personal projects: provider
|
|||||||
auth through egress and gitea.dideric.is over SSH.
|
auth through egress and gitea.dideric.is over SSH.
|
||||||
````
|
````
|
||||||
|
|
||||||
For a Codex-backed base bottle, set `agent_provider.template: codex`
|
For a Codex-backed base bottle, set `agent_provider.template: codex`.
|
||||||
and use the `codex_auth` egress role for the OpenAI API route. The
|
The Codex template expects ChatGPT/device login state instead of an
|
||||||
built-in Codex template uses `Dockerfile.codex`; set
|
`OPENAI_API_KEY` env var; no API-key placeholder is forwarded into the
|
||||||
`agent_provider.dockerfile` to build the agent from a custom
|
agent. To let bot-bottle read the host's current Codex ChatGPT access
|
||||||
Dockerfile while keeping the bot-bottle sidecars in place.
|
token and inject it from egress only for Codex's API calls, opt in
|
||||||
|
explicitly:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
agent_provider:
|
||||||
|
template: codex
|
||||||
|
forward_host_credentials: true
|
||||||
|
|
||||||
|
egress:
|
||||||
|
routes:
|
||||||
|
- host: auth.openai.com
|
||||||
|
path_allowlist:
|
||||||
|
- /api/accounts/deviceauth/
|
||||||
|
```
|
||||||
|
|
||||||
|
Run `codex login --device-auth` on the host before launch. The
|
||||||
|
launcher reads `tokens.access_token` from the host's
|
||||||
|
`~/.codex/auth.json`, verifies it is fresh user/device auth, and passes
|
||||||
|
it to the sidecar's `EGRESS_TOKEN_N` env slot. The agent container gets
|
||||||
|
a dummy `~/.codex/auth.json` that preserves the host auth-mode shape
|
||||||
|
but replaces credential values with placeholders. It keeps the selected
|
||||||
|
ChatGPT account id so Codex sends requests for the same account while
|
||||||
|
egress owns the real bearer token. The agent never receives real access
|
||||||
|
tokens, refresh tokens, or `OPENAI_API_KEY`. The effective egress table
|
||||||
|
automatically adds or upgrades `api.openai.com` and `chatgpt.com` to
|
||||||
|
authenticated routes when `forward_host_credentials` is true.
|
||||||
|
|
||||||
|
The built-in Codex template uses `Dockerfile.codex`; set
|
||||||
|
`agent_provider.dockerfile` to build the agent from a custom Dockerfile
|
||||||
|
while keeping the bot-bottle sidecars in place.
|
||||||
|
|
||||||
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
||||||
|
|
||||||
|
|||||||
@@ -4,14 +4,15 @@
|
|||||||
"env": {
|
"env": {
|
||||||
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
||||||
},
|
},
|
||||||
"git": [
|
"git-gate": {
|
||||||
{
|
"repos": {
|
||||||
"Name": "foo",
|
"foo": {
|
||||||
"Upstream": "ssh://git@upstream.invalid/path.git",
|
"url": "ssh://git@upstream.invalid/path.git",
|
||||||
"IdentityFile": "~/.cache/bot-bottle-demo/fake-key",
|
"identity": "~/.cache/bot-bottle-demo/fake-key",
|
||||||
"KnownHostKey": "ssh-ed25519 AAAAEXAMPLE"
|
"host_key": "ssh-ed25519 AAAAEXAMPLE"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
]
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
|||||||
@@ -7,14 +7,24 @@ command, default image, and prompt/auth behavior.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass
|
import json
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Literal
|
from typing import Literal
|
||||||
|
|
||||||
|
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
||||||
|
from .egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
||||||
|
|
||||||
|
|
||||||
PROVIDER_CLAUDE = "claude"
|
PROVIDER_CLAUDE = "claude"
|
||||||
PROVIDER_CODEX = "codex"
|
PROVIDER_CODEX = "codex"
|
||||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
|
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
|
||||||
|
|
||||||
|
# Hosts that egress injects the host ChatGPT bearer on when Codex
|
||||||
|
# forward_host_credentials is enabled. Pipelock must pass these through
|
||||||
|
# (no TLS MITM) or its header DLP blocks the injected JWT.
|
||||||
|
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
|
||||||
PromptMode = Literal["append_file", "read_prompt_file"]
|
PromptMode = Literal["append_file", "read_prompt_file"]
|
||||||
|
|
||||||
|
|
||||||
@@ -24,14 +34,68 @@ class AgentProviderRuntime:
|
|||||||
command: str
|
command: str
|
||||||
image: str
|
image: str
|
||||||
dockerfile: str
|
dockerfile: str
|
||||||
auth_role: str
|
|
||||||
placeholder_env: str
|
|
||||||
prompt_mode: PromptMode
|
prompt_mode: PromptMode
|
||||||
bypass_args: tuple[str, ...]
|
bypass_args: tuple[str, ...]
|
||||||
resume_args: tuple[str, ...]
|
resume_args: tuple[str, ...]
|
||||||
remote_control_args: tuple[str, ...]
|
remote_control_args: tuple[str, ...]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentProvisionDir:
|
||||||
|
guest_path: str
|
||||||
|
mode: str = "700"
|
||||||
|
owner: str = "node:node"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentProvisionFile:
|
||||||
|
host_path: Path
|
||||||
|
guest_path: str
|
||||||
|
mode: str = "600"
|
||||||
|
owner: str = "node:node"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentProvisionCommand:
|
||||||
|
argv: tuple[str, ...]
|
||||||
|
error: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentProvisionPlan:
|
||||||
|
"""Provider-owned guest setup.
|
||||||
|
|
||||||
|
Backends interpret this plan with their own copy/exec primitives.
|
||||||
|
Provider-specific content stays here so future provider plugins can
|
||||||
|
return the same shape without adding backend-plan fields.
|
||||||
|
|
||||||
|
`egress_routes` are provider-declared EgressRoutes that backends
|
||||||
|
pass to `Egress.prepare` and `PipelockProxy.prepare`. This keeps
|
||||||
|
provider logic out of the egress and pipelock modules — they merge
|
||||||
|
provider routes generically without knowing the provider type.
|
||||||
|
|
||||||
|
`hidden_env_names` is the set of env var names the provider injected
|
||||||
|
as non-secret placeholders. `print_util.visible_agent_env_names` uses
|
||||||
|
this to suppress them from the preflight summary so operators don't
|
||||||
|
mistake them for real credentials.
|
||||||
|
"""
|
||||||
|
|
||||||
|
template: str
|
||||||
|
command: str
|
||||||
|
prompt_mode: PromptMode
|
||||||
|
image: str
|
||||||
|
dockerfile: str
|
||||||
|
guest_env: dict[str, str]
|
||||||
|
env_vars: dict[str, str] = field(default_factory=dict)
|
||||||
|
dirs: tuple[AgentProvisionDir, ...] = ()
|
||||||
|
files: tuple[AgentProvisionFile, ...] = ()
|
||||||
|
pre_copy: tuple[AgentProvisionCommand, ...] = ()
|
||||||
|
verify: tuple[AgentProvisionCommand, ...] = ()
|
||||||
|
egress_routes: tuple[EgressRoute, ...] = ()
|
||||||
|
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
|
||||||
|
provisioned_env: dict[str, str] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
|
||||||
|
|
||||||
@@ -41,8 +105,6 @@ _RUNTIMES = {
|
|||||||
command="claude",
|
command="claude",
|
||||||
image="bot-bottle-claude:latest",
|
image="bot-bottle-claude:latest",
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||||
auth_role="claude_code_oauth",
|
|
||||||
placeholder_env="CLAUDE_CODE_OAUTH_TOKEN",
|
|
||||||
prompt_mode="append_file",
|
prompt_mode="append_file",
|
||||||
bypass_args=("--dangerously-skip-permissions",),
|
bypass_args=("--dangerously-skip-permissions",),
|
||||||
resume_args=("--continue",),
|
resume_args=("--continue",),
|
||||||
@@ -53,8 +115,6 @@ _RUNTIMES = {
|
|||||||
command="codex",
|
command="codex",
|
||||||
image="bot-bottle-codex:latest",
|
image="bot-bottle-codex:latest",
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||||
auth_role="codex_auth",
|
|
||||||
placeholder_env="OPENAI_API_KEY",
|
|
||||||
prompt_mode="read_prompt_file",
|
prompt_mode="read_prompt_file",
|
||||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||||
resume_args=("resume", "--last"),
|
resume_args=("resume", "--last"),
|
||||||
@@ -67,6 +127,126 @@ def runtime_for(template: str) -> AgentProviderRuntime:
|
|||||||
return _RUNTIMES[template]
|
return _RUNTIMES[template]
|
||||||
|
|
||||||
|
|
||||||
|
def agent_provision_plan(
|
||||||
|
*,
|
||||||
|
template: str,
|
||||||
|
dockerfile: str,
|
||||||
|
state_dir: Path,
|
||||||
|
guest_home: str = "/home/node",
|
||||||
|
guest_env: dict[str, str] | None = None,
|
||||||
|
auth_token: str = "",
|
||||||
|
forward_host_credentials: bool = False,
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
trusted_project_path: str = "",
|
||||||
|
) -> AgentProvisionPlan:
|
||||||
|
runtime = runtime_for(template)
|
||||||
|
resolved_guest_env = dict(guest_env or {})
|
||||||
|
trusted_path = trusted_project_path or guest_home
|
||||||
|
env_vars: dict[str, str] = {}
|
||||||
|
provisioned_env: dict[str, str] = {}
|
||||||
|
dirs: list[AgentProvisionDir] = []
|
||||||
|
files: list[AgentProvisionFile] = []
|
||||||
|
pre_copy: list[AgentProvisionCommand] = []
|
||||||
|
verify: list[AgentProvisionCommand] = []
|
||||||
|
egress_routes: list[EgressRoute] = []
|
||||||
|
hidden_env_names: frozenset[str] = frozenset()
|
||||||
|
|
||||||
|
if template == PROVIDER_CODEX:
|
||||||
|
env_vars["CODEX_CA_CERTIFICATE"] = "/etc/ssl/certs/ca-certificates.crt"
|
||||||
|
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
||||||
|
if forward_host_credentials:
|
||||||
|
env_vars["CODEX_HOME"] = auth_dir
|
||||||
|
dirs.append(AgentProvisionDir(auth_dir))
|
||||||
|
config_path = f"{auth_dir}/config.toml"
|
||||||
|
config_file = state_dir / "codex-config.toml"
|
||||||
|
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
||||||
|
config_file.write_text(
|
||||||
|
f'[projects."{toml_path}"]\n'
|
||||||
|
'trust_level = "trusted"\n'
|
||||||
|
)
|
||||||
|
config_file.chmod(0o600)
|
||||||
|
files.append(AgentProvisionFile(config_file, config_path))
|
||||||
|
|
||||||
|
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
||||||
|
egress_routes.append(EgressRoute(
|
||||||
|
host=host,
|
||||||
|
auth_scheme="Bearer" if forward_host_credentials else "",
|
||||||
|
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
||||||
|
tls_passthrough=True,
|
||||||
|
))
|
||||||
|
if forward_host_credentials:
|
||||||
|
_host_env = host_env or dict(os.environ)
|
||||||
|
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = codex_host_access_token(
|
||||||
|
_host_env,
|
||||||
|
)
|
||||||
|
auth_file = state_dir / "codex-auth.json"
|
||||||
|
write_codex_dummy_auth_file(auth_file, _host_env)
|
||||||
|
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
||||||
|
pre_copy.append(AgentProvisionCommand((
|
||||||
|
"find", auth_dir,
|
||||||
|
"-maxdepth", "1",
|
||||||
|
"-type", "f",
|
||||||
|
"(",
|
||||||
|
"-name", "*.sqlite",
|
||||||
|
"-o", "-name", "*.sqlite-*",
|
||||||
|
"-o", "-name", "*.codex-repair-*.bak",
|
||||||
|
")",
|
||||||
|
"-delete",
|
||||||
|
), "codex host credentials: could not reset runtime db files"))
|
||||||
|
verify.append(AgentProvisionCommand((
|
||||||
|
"runuser", "-u", "node", "--",
|
||||||
|
"env",
|
||||||
|
f"HOME={guest_home}",
|
||||||
|
f"CODEX_HOME={auth_dir}",
|
||||||
|
"codex", "login", "status",
|
||||||
|
), (
|
||||||
|
"codex host credentials: dummy auth was copied into the "
|
||||||
|
"guest, but Codex did not accept it"
|
||||||
|
)))
|
||||||
|
if template == PROVIDER_CLAUDE:
|
||||||
|
env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"] = "1"
|
||||||
|
env_vars["DISABLE_ERROR_REPORTING"] = "1"
|
||||||
|
claude_config = state_dir / "claude.json"
|
||||||
|
claude_projects = {
|
||||||
|
guest_home: {"hasTrustDialogAccepted": True},
|
||||||
|
}
|
||||||
|
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
||||||
|
claude_config.write_text(json.dumps({
|
||||||
|
"hasCompletedOnboarding": True,
|
||||||
|
"theme": "dark",
|
||||||
|
"bypassPermissionsModeAccepted": True,
|
||||||
|
"projects": claude_projects,
|
||||||
|
}, indent=2) + "\n")
|
||||||
|
claude_config.chmod(0o600)
|
||||||
|
files.append(AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"))
|
||||||
|
egress_routes.append(EgressRoute(
|
||||||
|
host="api.anthropic.com",
|
||||||
|
auth_scheme="Bearer" if auth_token else "",
|
||||||
|
token_ref=auth_token,
|
||||||
|
tls_passthrough=True,
|
||||||
|
))
|
||||||
|
if auth_token:
|
||||||
|
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||||
|
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||||
|
|
||||||
|
return AgentProvisionPlan(
|
||||||
|
template=template,
|
||||||
|
command=runtime.command,
|
||||||
|
prompt_mode=runtime.prompt_mode,
|
||||||
|
image=runtime.image,
|
||||||
|
dockerfile=dockerfile,
|
||||||
|
env_vars=env_vars,
|
||||||
|
guest_env=resolved_guest_env,
|
||||||
|
dirs=tuple(dirs),
|
||||||
|
files=tuple(files),
|
||||||
|
pre_copy=tuple(pre_copy),
|
||||||
|
verify=tuple(verify),
|
||||||
|
egress_routes=tuple(egress_routes),
|
||||||
|
hidden_env_names=hidden_env_names,
|
||||||
|
provisioned_env=provisioned_env,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def prompt_args(
|
def prompt_args(
|
||||||
prompt_mode: PromptMode,
|
prompt_mode: PromptMode,
|
||||||
prompt_path: str | None,
|
prompt_path: str | None,
|
||||||
|
|||||||
@@ -32,15 +32,22 @@ manifest does not carry a backend field; the host picks.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
import sys
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from contextlib import AbstractContextManager
|
from contextlib import AbstractContextManager
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Generic, Sequence, TypeVar
|
from typing import Any, Generic, Sequence, TypeVar
|
||||||
|
|
||||||
from ..log import die
|
from ..agent_provider import AgentProvisionPlan
|
||||||
|
from ..egress import EgressPlan
|
||||||
|
from ..git_gate import GitGatePlan
|
||||||
|
from ..log import die, info
|
||||||
from ..manifest import GitEntry, Manifest
|
from ..manifest import GitEntry, Manifest
|
||||||
|
from ..supervise import SupervisePlan
|
||||||
from ..util import expand_tilde
|
from ..util import expand_tilde
|
||||||
|
from ..workspace import WorkspacePlan
|
||||||
|
from .print_util import print_multi, visible_agent_env_names
|
||||||
from .util import host_skill_dir
|
from .util import host_skill_dir
|
||||||
|
|
||||||
|
|
||||||
@@ -65,15 +72,57 @@ class BottleSpec:
|
|||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class BottlePlan(ABC):
|
class BottlePlan(ABC):
|
||||||
"""Base output of a backend's prepare step. Concrete subclasses
|
"""Base output of a backend's prepare step. Concrete subclasses
|
||||||
(e.g. DockerBottlePlan) add backend-specific resolved fields and
|
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
|
||||||
implement `print`."""
|
|
||||||
|
|
||||||
spec: BottleSpec
|
spec: BottleSpec
|
||||||
stage_dir: Path
|
stage_dir: Path
|
||||||
|
git_gate_plan: GitGatePlan
|
||||||
|
egress_plan: EgressPlan
|
||||||
|
supervise_plan: SupervisePlan | None
|
||||||
|
agent_provision: AgentProvisionPlan
|
||||||
|
workspace_plan: WorkspacePlan
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def print(self, *, remote_control: bool) -> None:
|
def print(self, *, remote_control: bool) -> None:
|
||||||
"""Render the y/N preflight summary to stderr."""
|
"""Render the y/N preflight summary to stderr."""
|
||||||
|
del remote_control
|
||||||
|
spec = self.spec
|
||||||
|
manifest = spec.manifest
|
||||||
|
agent = manifest.agents[spec.agent_name]
|
||||||
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
|
|
||||||
|
env_names = visible_agent_env_names(
|
||||||
|
sorted(
|
||||||
|
set(bottle.env.keys())
|
||||||
|
| set(self.agent_provision.guest_env.keys())
|
||||||
|
),
|
||||||
|
hidden_env_names=self.agent_provision.hidden_env_names,
|
||||||
|
)
|
||||||
|
|
||||||
|
print(file=sys.stderr)
|
||||||
|
info(f"agent : {spec.agent_name}")
|
||||||
|
info(f"provider : {self.agent_provision.template}")
|
||||||
|
print_multi("env ", env_names)
|
||||||
|
print_multi("skills ", list(agent.skills))
|
||||||
|
info(f"bottle : {agent.bottle}")
|
||||||
|
|
||||||
|
identity = manifest.git_identity_summary(spec.agent_name)
|
||||||
|
if identity:
|
||||||
|
info(f" git identity : {identity}")
|
||||||
|
|
||||||
|
git_lines = [
|
||||||
|
f"{u.name} → {u.upstream_host}:{u.upstream_port}"
|
||||||
|
for u in self.git_gate_plan.upstreams
|
||||||
|
]
|
||||||
|
if git_lines:
|
||||||
|
print_multi(" git gate ", git_lines)
|
||||||
|
|
||||||
|
if self.egress_plan.routes:
|
||||||
|
egress_lines = []
|
||||||
|
for r in self.egress_plan.routes:
|
||||||
|
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||||
|
egress_lines.append(f"{r.host}{auth}")
|
||||||
|
print_multi(" egress ", egress_lines)
|
||||||
|
print(file=sys.stderr)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -273,7 +322,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
decide whether to add provider-specific prompt args to the agent's
|
decide whether to add provider-specific prompt args to the agent's
|
||||||
argv.
|
argv.
|
||||||
|
|
||||||
Default orchestration: ca → prompt → skills → git →
|
Default orchestration: ca → prompt → skills → workspace → git →
|
||||||
supervise. CA install runs first so the agent's trust store
|
supervise. CA install runs first so the agent's trust store
|
||||||
is rebuilt before anything inside the agent makes a TLS call.
|
is rebuilt before anything inside the agent makes a TLS call.
|
||||||
Subclasses typically don't override this; they implement the
|
Subclasses typically don't override this; they implement the
|
||||||
@@ -286,7 +335,9 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
intercepted without per-tool reconfiguration."""
|
intercepted without per-tool reconfiguration."""
|
||||||
self.provision_ca(plan, target)
|
self.provision_ca(plan, target)
|
||||||
prompt_path = self.provision_prompt(plan, target)
|
prompt_path = self.provision_prompt(plan, target)
|
||||||
|
self.provision_provider_auth(plan, target)
|
||||||
self.provision_skills(plan, target)
|
self.provision_skills(plan, target)
|
||||||
|
self.provision_workspace(plan, target)
|
||||||
self.provision_git(plan, target)
|
self.provision_git(plan, target)
|
||||||
self.provision_supervise(plan, target)
|
self.provision_supervise(plan, target)
|
||||||
return prompt_path
|
return prompt_path
|
||||||
@@ -300,6 +351,11 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
backend overrides to docker-cp the cert in and run
|
backend overrides to docker-cp the cert in and run
|
||||||
`update-ca-certificates`."""
|
`update-ca-certificates`."""
|
||||||
|
|
||||||
|
def provision_provider_auth(self, plan: PlanT, target: str) -> None:
|
||||||
|
"""Install non-secret provider auth marker files into the agent
|
||||||
|
home when a provider needs them to select the right auth mode.
|
||||||
|
The default is no-op."""
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
||||||
"""Copy the prompt file into the running bottle. Returns the
|
"""Copy the prompt file into the running bottle. Returns the
|
||||||
@@ -312,6 +368,11 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
"""Copy the agent's named skills from the host into the
|
"""Copy the agent's named skills from the host into the
|
||||||
running bottle. No-op when the agent has no skills."""
|
running bottle. No-op when the agent has no skills."""
|
||||||
|
|
||||||
|
def provision_workspace(self, plan: PlanT, target: str) -> None:
|
||||||
|
"""Copy the operator workspace into the running bottle when
|
||||||
|
the backend cannot bake it into the agent image. Default is
|
||||||
|
no-op for backends like Docker that handle this before launch."""
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def provision_git(self, plan: PlanT, target: str) -> None:
|
def provision_git(self, plan: PlanT, target: str) -> None:
|
||||||
"""Copy the host's cwd `.git` directory into the running
|
"""Copy the host's cwd `.git` directory into the running
|
||||||
@@ -413,14 +474,20 @@ def enumerate_active_agents() -> list[ActiveAgent]:
|
|||||||
"""All currently-running agents, across every available
|
"""All currently-running agents, across every available
|
||||||
backend. Used by CLI `list active` and the dashboard's agents
|
backend. Used by CLI `list active` and the dashboard's agents
|
||||||
pane so neither has to know which backends exist. Skips
|
pane so neither has to know which backends exist. Skips
|
||||||
backends whose `is_available()` reports False. Ordered by
|
backends whose `is_available()` reports False.
|
||||||
backend name, then by whatever each backend's
|
|
||||||
`enumerate_active` returns."""
|
Sorted by `(started_at, slug)` so the list is stable across
|
||||||
|
dashboard refresh ticks — agents don't shift position while
|
||||||
|
the operator navigates with arrow keys. ISO 8601 timestamps
|
||||||
|
sort lexicographically in chronological order; `slug` is the
|
||||||
|
deterministic tiebreaker. Agents with missing metadata
|
||||||
|
(`started_at == ""`) sort first."""
|
||||||
out: list[ActiveAgent] = []
|
out: list[ActiveAgent] = []
|
||||||
for name in known_backend_names():
|
for name in known_backend_names():
|
||||||
if not has_backend(name):
|
if not has_backend(name):
|
||||||
continue
|
continue
|
||||||
out.extend(_BACKENDS[name].enumerate_active())
|
out.extend(_BACKENDS[name].enumerate_active())
|
||||||
|
out.sort(key=lambda a: (a.started_at, a.slug))
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ from .bottle_plan import DockerBottlePlan
|
|||||||
from .provision import ca as _ca
|
from .provision import ca as _ca
|
||||||
from .provision import git as _git
|
from .provision import git as _git
|
||||||
from .provision import prompt as _prompt
|
from .provision import prompt as _prompt
|
||||||
|
from .provision import provider_auth as _provider_auth
|
||||||
from .provision import skills as _skills
|
from .provision import skills as _skills
|
||||||
from .provision import supervise as _supervise_prov
|
from .provision import supervise as _supervise_prov
|
||||||
|
|
||||||
@@ -62,6 +63,9 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
|||||||
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
||||||
return _prompt.provision_prompt(plan, target)
|
return _prompt.provision_prompt(plan, target)
|
||||||
|
|
||||||
|
def provision_provider_auth(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
_provider_auth.provision_provider_auth(plan, target)
|
||||||
|
|
||||||
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
_skills.provision_skills(plan, target)
|
_skills.provision_skills(plan, target)
|
||||||
|
|
||||||
|
|||||||
@@ -2,30 +2,25 @@
|
|||||||
|
|
||||||
Carries the Docker-specific resolved fields produced by
|
Carries the Docker-specific resolved fields produced by
|
||||||
DockerBottleBackend.prepare. The launch step consumes it without
|
DockerBottleBackend.prepare. The launch step consumes it without
|
||||||
further resolution; show_plan-style rendering is the `print` method.
|
further resolution; preflight rendering is inherited from BottlePlan.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import sys
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import PromptMode
|
||||||
from ...egress import EgressPlan
|
|
||||||
from ...git_gate import GitGatePlan
|
|
||||||
from ...log import info
|
|
||||||
from ...pipelock import PipelockProxyPlan
|
from ...pipelock import PipelockProxyPlan
|
||||||
from ...supervise import SupervisePlan
|
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
from ..print_util import print_multi, visible_agent_env_names
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class DockerBottlePlan(BottlePlan):
|
class DockerBottlePlan(BottlePlan):
|
||||||
"""Docker-specific resolved fields produced by
|
"""Docker-specific resolved fields produced by
|
||||||
DockerBottleBackend.prepare. Inherits `spec` and `stage_dir` from
|
DockerBottleBackend.prepare. Inherits `spec`, `stage_dir`,
|
||||||
BottlePlan."""
|
`git_gate_plan`, `egress_plan`, `supervise_plan`, and
|
||||||
|
`agent_provision` from BottlePlan."""
|
||||||
|
|
||||||
slug: str
|
slug: str
|
||||||
container_name: str
|
container_name: str
|
||||||
@@ -46,60 +41,16 @@ class DockerBottlePlan(BottlePlan):
|
|||||||
forwarded_env: dict[str, str] = field(repr=False)
|
forwarded_env: dict[str, str] = field(repr=False)
|
||||||
prompt_file: Path
|
prompt_file: Path
|
||||||
proxy_plan: PipelockProxyPlan
|
proxy_plan: PipelockProxyPlan
|
||||||
git_gate_plan: GitGatePlan
|
|
||||||
egress_plan: EgressPlan
|
|
||||||
# None when bottle.supervise is False. PRD 0013 supervise sidecar
|
|
||||||
# is opt-in via the manifest's bottle.supervise field.
|
|
||||||
supervise_plan: SupervisePlan | None
|
|
||||||
use_runsc: bool
|
use_runsc: bool
|
||||||
agent_command: str = "claude"
|
|
||||||
agent_prompt_mode: PromptMode = "append_file"
|
|
||||||
agent_provider_template: str = "claude"
|
|
||||||
|
|
||||||
def print(self, *, remote_control: bool) -> None:
|
@property
|
||||||
"""Render the y/N preflight summary to stderr — compact form
|
def agent_command(self) -> str:
|
||||||
intended to fit on screen without scrolling. The full
|
return self.agent_provision.command
|
||||||
structured shape (image, container, runtime, etc.) lives on
|
|
||||||
this dataclass for tooling that wants to introspect it."""
|
|
||||||
del remote_control # not surfaced in the compact summary
|
|
||||||
spec = self.spec
|
|
||||||
manifest = spec.manifest
|
|
||||||
agent = manifest.agents[spec.agent_name]
|
|
||||||
bottle = manifest.bottle_for(spec.agent_name)
|
|
||||||
# The agent sees the union of literal env names (rendered into
|
|
||||||
# --env-file) and forwarded env names (`-e NAME` with the
|
|
||||||
# value arriving via subprocess env). The forwarded set holds
|
|
||||||
# the OAuth token (CLAUDE_CODE_OAUTH_TOKEN) and any host-env
|
|
||||||
# interpolations from the manifest; egress holds
|
|
||||||
# upstream tokens in its own environ, so no token forwarding
|
|
||||||
# from the agent to the proxy is needed.
|
|
||||||
env_names = visible_agent_env_names(
|
|
||||||
sorted(set(bottle.env.keys()) | set(self.forwarded_env.keys())),
|
|
||||||
agent_provider_template=self.agent_provider_template,
|
|
||||||
)
|
|
||||||
|
|
||||||
print(file=sys.stderr)
|
@property
|
||||||
info(f"agent : {spec.agent_name}")
|
def agent_prompt_mode(self) -> PromptMode:
|
||||||
info(f"provider : {self.agent_provider_template}")
|
return self.agent_provision.prompt_mode
|
||||||
print_multi("env ", env_names)
|
|
||||||
print_multi("skills ", list(agent.skills))
|
|
||||||
info(f"bottle : {agent.bottle}")
|
|
||||||
|
|
||||||
identity = manifest.git_identity_summary(spec.agent_name)
|
@property
|
||||||
if identity:
|
def agent_provider_template(self) -> str:
|
||||||
info(f" git identity : {identity}")
|
return self.agent_provision.template
|
||||||
|
|
||||||
git_lines = [
|
|
||||||
f"{u.upstream_host}:{u.upstream_port}"
|
|
||||||
for u in self.git_gate_plan.upstreams
|
|
||||||
]
|
|
||||||
if git_lines:
|
|
||||||
print_multi(" git gate ", git_lines)
|
|
||||||
|
|
||||||
if self.egress_plan.routes:
|
|
||||||
egress_lines = []
|
|
||||||
for r in self.egress_plan.routes:
|
|
||||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
|
||||||
egress_lines.append(f"{r.host}{auth}")
|
|
||||||
print_multi(" egress ", egress_lines)
|
|
||||||
print(file=sys.stderr)
|
|
||||||
|
|||||||
@@ -105,6 +105,10 @@ class BottleMetadata:
|
|||||||
# written before chunk 3 (resume / inspect should fall back to
|
# written before chunk 3 (resume / inspect should fall back to
|
||||||
# deriving from identity in that case).
|
# deriving from identity in that case).
|
||||||
compose_project: str = ""
|
compose_project: str = ""
|
||||||
|
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
|
||||||
|
# for state dirs written before PRD 0040; callers default to "docker"
|
||||||
|
# for backward compatibility.
|
||||||
|
backend: str = ""
|
||||||
|
|
||||||
|
|
||||||
def metadata_path(identity: str) -> Path:
|
def metadata_path(identity: str) -> Path:
|
||||||
@@ -138,6 +142,7 @@ def read_metadata(identity: str) -> BottleMetadata | None:
|
|||||||
copy_cwd=bool(raw.get("copy_cwd", False)),
|
copy_cwd=bool(raw.get("copy_cwd", False)),
|
||||||
started_at=str(raw.get("started_at", "")),
|
started_at=str(raw.get("started_at", "")),
|
||||||
compose_project=str(raw.get("compose_project", "")),
|
compose_project=str(raw.get("compose_project", "")),
|
||||||
|
backend=str(raw.get("backend", "")),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ from ...egress import (
|
|||||||
EGRESS_HOSTNAME,
|
EGRESS_HOSTNAME,
|
||||||
EGRESS_ROUTES_IN_CONTAINER,
|
EGRESS_ROUTES_IN_CONTAINER,
|
||||||
)
|
)
|
||||||
from ...git_gate import GIT_GATE_HOSTNAME, git_gate_aggregate_extra_hosts
|
from ...git_gate import GIT_GATE_HOSTNAME
|
||||||
from ...log import die, warn
|
from ...log import die, warn
|
||||||
from ...pipelock import PIPELOCK_HOSTNAME
|
from ...pipelock import PIPELOCK_HOSTNAME
|
||||||
from ...supervise import (
|
from ...supervise import (
|
||||||
@@ -198,7 +198,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
env.append(token_env)
|
env.append(token_env)
|
||||||
|
|
||||||
# --- git-gate ----------------------------------------------------
|
# --- git-gate ----------------------------------------------------
|
||||||
extra_hosts: list[str] = []
|
|
||||||
gp = plan.git_gate_plan
|
gp = plan.git_gate_plan
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
volumes += [
|
volumes += [
|
||||||
@@ -217,8 +216,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
u.known_hosts_file,
|
u.known_hosts_file,
|
||||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||||
))
|
))
|
||||||
extra_map = git_gate_aggregate_extra_hosts(gp.upstreams)
|
|
||||||
extra_hosts = [f"{host}:{ip}" for host, ip in sorted(extra_map.items())]
|
|
||||||
|
|
||||||
# --- supervise ---------------------------------------------------
|
# --- supervise ---------------------------------------------------
|
||||||
sp = plan.supervise_plan
|
sp = plan.supervise_plan
|
||||||
@@ -261,8 +258,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
"environment": env,
|
"environment": env,
|
||||||
"volumes": volumes,
|
"volumes": volumes,
|
||||||
}
|
}
|
||||||
if extra_hosts:
|
|
||||||
service["extra_hosts"] = extra_hosts
|
|
||||||
return service
|
return service
|
||||||
|
|
||||||
|
|
||||||
@@ -286,6 +281,8 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
||||||
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
||||||
]
|
]
|
||||||
|
for name, value in sorted(plan.agent_provision.guest_env.items()):
|
||||||
|
env.append(f"{name}={value}")
|
||||||
# Forwarded vars (OAuth token, manifest host-interpolations):
|
# Forwarded vars (OAuth token, manifest host-interpolations):
|
||||||
# bare name → inherits from compose-up process env, value
|
# bare name → inherits from compose-up process env, value
|
||||||
# never lands on argv or in the compose file.
|
# never lands on argv or in the compose file.
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ from pathlib import Path
|
|||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
|
|
||||||
from ...egress import egress_resolve_token_values
|
from ...egress import egress_resolve_token_values
|
||||||
from ...log import info
|
from ...log import info, warn
|
||||||
from . import network as network_mod
|
from . import network as network_mod
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
from .bottle import DockerBottle
|
from .bottle import DockerBottle
|
||||||
@@ -87,10 +87,11 @@ def launch(
|
|||||||
def teardown() -> None:
|
def teardown() -> None:
|
||||||
try:
|
try:
|
||||||
stack.close()
|
stack.close()
|
||||||
except BaseException:
|
except BaseException as exc:
|
||||||
# Teardown must not raise; swallow so the caller's
|
warn(
|
||||||
# __exit__ path can still propagate the original error.
|
f"teardown failed for container {plan.container_name}"
|
||||||
pass
|
f" (compose-down): {exc!r}"
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Step 1: agent image build. Sidecar images get built lazily by
|
# Step 1: agent image build. Sidecar images get built lazily by
|
||||||
@@ -101,7 +102,7 @@ def launch(
|
|||||||
)
|
)
|
||||||
if plan.derived_image:
|
if plan.derived_image:
|
||||||
docker_mod.build_image_with_cwd(
|
docker_mod.build_image_with_cwd(
|
||||||
plan.derived_image, plan.image, plan.spec.user_cwd
|
plan.derived_image, plan.image, plan.workspace_plan
|
||||||
)
|
)
|
||||||
|
|
||||||
# Networks: compose-managed. The names are derived
|
# Networks: compose-managed. The names are derived
|
||||||
@@ -176,11 +177,10 @@ def launch(
|
|||||||
# Step 7: compose up. Token values + the OAuth placeholder
|
# Step 7: compose up. Token values + the OAuth placeholder
|
||||||
# flow through subprocess env; the compose file holds only
|
# flow through subprocess env; the compose file holds only
|
||||||
# bare names for the secret-carrying entries.
|
# bare names for the secret-carrying entries.
|
||||||
token_values: dict[str, str] = {}
|
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||||
if plan.egress_plan.routes:
|
token_values = egress_resolve_token_values(
|
||||||
token_values = egress_resolve_token_values(
|
plan.egress_plan.token_env_map, effective_env,
|
||||||
plan.egress_plan.token_env_map, dict(os.environ),
|
)
|
||||||
)
|
|
||||||
compose_env: dict[str, str] = {
|
compose_env: dict[str, str] = {
|
||||||
**os.environ,
|
**os.environ,
|
||||||
**plan.forwarded_env,
|
**plan.forwarded_env,
|
||||||
|
|||||||
@@ -12,15 +12,17 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
|
from dataclasses import replace
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import runtime_for
|
from ...agent_provider import agent_provision_plan, runtime_for
|
||||||
from ...egress import Egress
|
from ...egress import Egress
|
||||||
from ...env import ResolvedEnv, resolve_env
|
from ...env import ResolvedEnv, resolve_env
|
||||||
from ...git_gate import GitGate
|
from ...git_gate import GitGate
|
||||||
from ...log import die
|
from ...log import die
|
||||||
from ...pipelock import PipelockProxy
|
from ...pipelock import PipelockProxy
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
|
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||||
from .. import BottleSpec
|
from .. import BottleSpec
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
from .bottle_plan import DockerBottlePlan
|
from .bottle_plan import DockerBottlePlan
|
||||||
@@ -61,6 +63,8 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
|
guest_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||||
|
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||||
|
|
||||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||||
# mints a random-suffixed identity (so parallel runs of the same
|
# mints a random-suffixed identity (so parallel runs of the same
|
||||||
@@ -78,6 +82,7 @@ def resolve_plan(
|
|||||||
copy_cwd=spec.copy_cwd,
|
copy_cwd=spec.copy_cwd,
|
||||||
started_at=datetime.now(timezone.utc).isoformat(),
|
started_at=datetime.now(timezone.utc).isoformat(),
|
||||||
compose_project=f"bot-bottle-{slug}",
|
compose_project=f"bot-bottle-{slug}",
|
||||||
|
backend="docker",
|
||||||
))
|
))
|
||||||
# Clear any leftover preserve marker from a prior capability-block
|
# Clear any leftover preserve marker from a prior capability-block
|
||||||
# so this fresh launch can be cleaned up at session-end unless
|
# so this fresh launch can be cleaned up at session-end unless
|
||||||
@@ -158,17 +163,45 @@ def resolve_plan(
|
|||||||
prompt_file.write_text("")
|
prompt_file.write_text("")
|
||||||
prompt_file.chmod(0o600)
|
prompt_file.chmod(0o600)
|
||||||
|
|
||||||
pipelock_dir = pipelock_state_dir(slug)
|
|
||||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
proxy_plan = proxy.prepare(bottle, slug, pipelock_dir)
|
|
||||||
|
|
||||||
git_gate_dir = git_gate_state_dir(slug)
|
git_gate_dir = git_gate_state_dir(slug)
|
||||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||||
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
|
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
|
||||||
|
|
||||||
|
resolved = resolve_env(manifest, spec.agent_name)
|
||||||
|
# Everything that should reach the bottle by-name (so its value
|
||||||
|
# never lands on argv or in env_file) goes into one dict. Nothing
|
||||||
|
# mutates the host os.environ.
|
||||||
|
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
||||||
|
_write_env_file(resolved, env_file)
|
||||||
|
prompt_file.write_text(agent.prompt)
|
||||||
|
|
||||||
|
use_runsc = docker_mod.runsc_available()
|
||||||
|
agent_provision = agent_provision_plan(
|
||||||
|
template=provider.template,
|
||||||
|
dockerfile=dockerfile_path,
|
||||||
|
state_dir=agent_dir,
|
||||||
|
guest_home=guest_home,
|
||||||
|
forward_host_credentials=provider.forward_host_credentials,
|
||||||
|
auth_token=provider.auth_token,
|
||||||
|
host_env=dict(os.environ),
|
||||||
|
trusted_project_path=workspace_plan.workdir,
|
||||||
|
)
|
||||||
|
guest_env = dict(agent_provision.guest_env)
|
||||||
|
for key, val in agent_provision.env_vars.items():
|
||||||
|
guest_env.setdefault(key, val)
|
||||||
|
agent_provision = replace(agent_provision, guest_env=guest_env)
|
||||||
|
|
||||||
|
pipelock_dir = pipelock_state_dir(slug)
|
||||||
|
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
proxy_plan = proxy.prepare(
|
||||||
|
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
||||||
|
)
|
||||||
|
|
||||||
egress_dir = egress_state_dir(slug)
|
egress_dir = egress_state_dir(slug)
|
||||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||||
egress_plan = egress.prepare(bottle, slug, egress_dir)
|
egress_plan = egress.prepare(
|
||||||
|
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||||
|
)
|
||||||
|
|
||||||
supervise_plan = None
|
supervise_plan = None
|
||||||
if bottle.supervise:
|
if bottle.supervise:
|
||||||
@@ -196,33 +229,6 @@ def resolve_plan(
|
|||||||
slug, supervise_dir,
|
slug, supervise_dir,
|
||||||
dockerfile_content=dockerfile_content,
|
dockerfile_content=dockerfile_content,
|
||||||
)
|
)
|
||||||
resolved = resolve_env(manifest, spec.agent_name)
|
|
||||||
# Everything that should reach the bottle by-name (so its value
|
|
||||||
# never lands on argv or in env_file) goes into one dict. Nothing
|
|
||||||
# mutates the host os.environ.
|
|
||||||
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
|
||||||
# When the bottle declares an egress route with the
|
|
||||||
# `claude_code_oauth` role marker, claude-code's outbound
|
|
||||||
# Authorization gets stripped + re-injected by egress. The
|
|
||||||
# agent's environ still needs *something* claude-code recognises
|
|
||||||
# as a credential or it refuses to start; ship a non-secret
|
|
||||||
# placeholder. The placeholder isn't any real token value, so
|
|
||||||
# leaking it would tell an attacker only that egress is in
|
|
||||||
# front. Manifest validation enforces singleton on this role.
|
|
||||||
has_provider_auth = any(
|
|
||||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
|
||||||
)
|
|
||||||
if has_provider_auth:
|
|
||||||
forwarded_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
|
||||||
if provider.template == "claude" and has_provider_auth:
|
|
||||||
# Belt-and-braces: turn off telemetry endpoints (statsig,
|
|
||||||
# error reporting) that egress can't gate by auth.
|
|
||||||
forwarded_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
|
||||||
forwarded_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
|
||||||
_write_env_file(resolved, env_file)
|
|
||||||
prompt_file.write_text(agent.prompt)
|
|
||||||
|
|
||||||
use_runsc = docker_mod.runsc_available()
|
|
||||||
|
|
||||||
return DockerBottlePlan(
|
return DockerBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
@@ -242,9 +248,8 @@ def resolve_plan(
|
|||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
use_runsc=use_runsc,
|
use_runsc=use_runsc,
|
||||||
agent_command=provider_runtime.command,
|
agent_provision=agent_provision,
|
||||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
workspace_plan=workspace_plan,
|
||||||
agent_provider_template=provider.template,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
Three concerns, all about git in the agent:
|
Three concerns, all about git in the agent:
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
||||||
into /home/node/workspace/.git so the agent operates on the
|
into the planned guest workspace so the agent operates on the
|
||||||
user's repo.
|
user's repo.
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
~/.gitconfig with insteadOf rules so every git operation
|
||||||
@@ -20,7 +20,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
import subprocess
|
import subprocess
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||||
from ....log import info
|
from ....log import info
|
||||||
@@ -40,19 +39,22 @@ def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
|
|||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||||
it into /home/node/workspace/.git and fix ownership. No-op
|
it into /home/node/workspace/.git and fix ownership. No-op
|
||||||
otherwise."""
|
otherwise."""
|
||||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
workspace = plan.workspace_plan
|
||||||
|
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
||||||
return
|
return
|
||||||
container = target
|
container = target
|
||||||
info(f"copying {plan.spec.user_cwd}/.git -> {container}:/home/node/workspace/.git")
|
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||||
|
host_git = str(workspace.host_path / ".git")
|
||||||
|
info(f"copying {host_git} -> {container}:{guest_workspace_git}")
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["docker", "cp", f"{plan.spec.user_cwd}/.git", f"{container}:/home/node/workspace/.git"],
|
["docker", "cp", host_git, f"{container}:{guest_workspace_git}"],
|
||||||
stdout=subprocess.DEVNULL,
|
stdout=subprocess.DEVNULL,
|
||||||
check=True,
|
check=True,
|
||||||
)
|
)
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
[
|
[
|
||||||
"docker", "exec", "-u", "0", container,
|
"docker", "exec", "-u", "0", container,
|
||||||
"chown", "-R", "node:node", "/home/node/workspace/.git",
|
"chown", "-R", workspace.owner, guest_workspace_git,
|
||||||
],
|
],
|
||||||
stdout=subprocess.DEVNULL,
|
stdout=subprocess.DEVNULL,
|
||||||
check=True,
|
check=True,
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
"""Provision non-secret provider auth markers into a Docker bottle."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_provider_auth(plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
"""Apply provider-owned guest setup through Docker primitives."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
_exec(target, ["mkdir", "-p", d.guest_path])
|
||||||
|
_exec(target, ["chown", d.owner, d.guest_path])
|
||||||
|
_exec(target, ["chmod", d.mode, d.guest_path])
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(target, list(command.argv))
|
||||||
|
for f in provision.files:
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "cp", str(f.host_path), f"{target}:{f.guest_path}"],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
_exec(target, ["chown", f.owner, f.guest_path])
|
||||||
|
_exec(target, ["chmod", f.mode, f.guest_path])
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(target, list(command.argv))
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(target: str, argv: list[str]) -> None:
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", target, *argv],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
@@ -7,9 +7,11 @@ from __future__ import annotations
|
|||||||
import re
|
import re
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
|
import tempfile
|
||||||
from typing import Iterable, Iterator
|
from typing import Iterable, Iterator
|
||||||
|
|
||||||
from ...log import die, info
|
from ...log import die, info
|
||||||
|
from ...workspace import WorkspacePlan
|
||||||
|
|
||||||
|
|
||||||
# Cap on the suffix the container-name conflict logic will try before
|
# Cap on the suffix the container-name conflict logic will try before
|
||||||
@@ -116,35 +118,39 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
|||||||
subprocess.run(args, check=True)
|
subprocess.run(args, check=True)
|
||||||
|
|
||||||
|
|
||||||
_TRUST_DIALOG_NODE_SCRIPT = (
|
def build_image_with_cwd(
|
||||||
'const fs=require("fs"),p=process.env.HOME+"/.claude.json",'
|
derived: str,
|
||||||
'c=JSON.parse(fs.readFileSync(p,"utf8"));'
|
base: str,
|
||||||
'c.projects=c.projects||{};'
|
workspace: WorkspacePlan,
|
||||||
'c.projects[process.env.HOME+"/workspace"]={hasTrustDialogAccepted:true};'
|
) -> None:
|
||||||
'fs.writeFileSync(p,JSON.stringify(c,null,2));'
|
"""Build a thin derived image that copies the workspace into
|
||||||
)
|
the plan's guest path and sets the plan's workdir."""
|
||||||
|
|
||||||
|
|
||||||
def build_image_with_cwd(derived: str, base: str, cwd: str) -> None:
|
|
||||||
"""Build a thin derived image that copies <cwd> into
|
|
||||||
/home/node/workspace and adds a trust-dialog entry for it."""
|
|
||||||
import os
|
import os
|
||||||
|
|
||||||
|
cwd = str(workspace.host_path)
|
||||||
if not os.path.isdir(cwd):
|
if not os.path.isdir(cwd):
|
||||||
die(f"cwd not found at {cwd}")
|
die(f"cwd not found at {cwd}")
|
||||||
info(f"building image {derived} from {base} with {cwd} -> /home/node/workspace")
|
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
||||||
dockerfile = (
|
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
||||||
f"FROM {base}\n"
|
context_dir = os.path.join(tmp, "context")
|
||||||
f"COPY --chown=node:node . /home/node/workspace\n"
|
staged_workspace = os.path.join(context_dir, "workspace")
|
||||||
f"RUN node -e '{_TRUST_DIALOG_NODE_SCRIPT}'\n"
|
shutil.copytree(
|
||||||
f"WORKDIR /home/node/workspace\n"
|
cwd,
|
||||||
)
|
staged_workspace,
|
||||||
subprocess.run(
|
symlinks=True,
|
||||||
["docker", "build", "-t", derived, "-f", "-", cwd],
|
ignore=shutil.ignore_patterns(".git"),
|
||||||
input=dockerfile,
|
)
|
||||||
text=True,
|
dockerfile = (
|
||||||
check=True,
|
f"FROM {base}\n"
|
||||||
)
|
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
||||||
|
f"WORKDIR {workspace.workdir}\n"
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "build", "-t", derived, "-f", "-", context_dir],
|
||||||
|
input=dockerfile,
|
||||||
|
text=True,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def image_id(ref: str) -> str:
|
def image_id(ref: str) -> str:
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from typing import Sequence
|
from typing import Sequence
|
||||||
|
|
||||||
from ..agent_provider import runtime_for
|
|
||||||
from ..log import info
|
from ..log import info
|
||||||
|
|
||||||
|
|
||||||
@@ -30,16 +29,13 @@ def print_multi(label: str, values: Sequence[str]) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def visible_agent_env_names(
|
def visible_agent_env_names(
|
||||||
env_names: Sequence[str], *, agent_provider_template: str,
|
env_names: Sequence[str], *, hidden_env_names: frozenset[str],
|
||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
"""Env names worth showing in launch summaries.
|
"""Env names worth showing in launch summaries.
|
||||||
|
|
||||||
Provider auth placeholders (`OPENAI_API_KEY`,
|
Provider-injected placeholder env vars are implementation details:
|
||||||
`CLAUDE_CODE_OAUTH_TOKEN`) are implementation details: they are
|
they are non-secret dummy values that satisfy provider CLIs while
|
||||||
non-secret dummy values that satisfy the provider CLI while egress
|
egress injects the real Authorization header. The plan's
|
||||||
injects the real upstream Authorization header. Showing them in
|
`hidden_env_names` carries exactly which names to suppress.
|
||||||
preflight makes the operator think a real key is entering the
|
|
||||||
agent, so hide only that provider-owned placeholder.
|
|
||||||
"""
|
"""
|
||||||
hidden = {runtime_for(agent_provider_template).placeholder_env}
|
return sorted({name for name in env_names if name and name not in hidden_env_names})
|
||||||
return sorted({name for name in env_names if name not in hidden})
|
|
||||||
|
|||||||
@@ -19,8 +19,10 @@ from .bottle_plan import SmolmachinesBottlePlan
|
|||||||
from .provision import ca as _ca
|
from .provision import ca as _ca
|
||||||
from .provision import git as _git
|
from .provision import git as _git
|
||||||
from .provision import prompt as _prompt
|
from .provision import prompt as _prompt
|
||||||
|
from .provision import provider_auth as _provider_auth
|
||||||
from .provision import skills as _skills
|
from .provision import skills as _skills
|
||||||
from .provision import supervise as _supervise
|
from .provision import supervise as _supervise
|
||||||
|
from .provision import workspace as _workspace
|
||||||
|
|
||||||
|
|
||||||
class SmolmachinesBottleBackend(
|
class SmolmachinesBottleBackend(
|
||||||
@@ -61,11 +63,21 @@ class SmolmachinesBottleBackend(
|
|||||||
) -> str | None:
|
) -> str | None:
|
||||||
return _prompt.provision_prompt(plan, target)
|
return _prompt.provision_prompt(plan, target)
|
||||||
|
|
||||||
|
def provision_provider_auth(
|
||||||
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
|
) -> None:
|
||||||
|
_provider_auth.provision_provider_auth(plan, target)
|
||||||
|
|
||||||
def provision_skills(
|
def provision_skills(
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
) -> None:
|
) -> None:
|
||||||
_skills.provision_skills(plan, target)
|
_skills.provision_skills(plan, target)
|
||||||
|
|
||||||
|
def provision_workspace(
|
||||||
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
|
) -> None:
|
||||||
|
_workspace.provision_workspace(plan, target)
|
||||||
|
|
||||||
def provision_git(
|
def provision_git(
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
@@ -45,19 +45,11 @@ _HOME_FOR = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def _env_flags_for(user: str) -> list[str]:
|
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||||
return ["-e", f"HOME={home}", "-e", f"USER={user}"]
|
out = [f"HOME={home}", f"USER={user}"]
|
||||||
|
|
||||||
|
|
||||||
def _guest_env_flags(env: Mapping[str, str]) -> list[str]:
|
|
||||||
"""Render `{K: V}` into a flat `-e K=V` argv slice for
|
|
||||||
`smolvm machine exec`. `smolvm machine create -e` set env
|
|
||||||
on PID 1 but it doesn't propagate to fresh exec process
|
|
||||||
trees, so we have to re-pass them every call."""
|
|
||||||
out: list[str] = []
|
|
||||||
for k, v in env.items():
|
for k, v in env.items():
|
||||||
out += ["-e", f"{k}={v}"]
|
out.append(f"{k}={v}")
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
@@ -98,9 +90,8 @@ class SmolmachinesBottle(Bottle):
|
|||||||
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
||||||
if tty:
|
if tty:
|
||||||
flags += ["-i", "-t"]
|
flags += ["-i", "-t"]
|
||||||
flags += _env_flags_for("node")
|
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
|
||||||
flags += _guest_env_flags(self._guest_env)
|
self.agent_command]
|
||||||
agent_tail = [self.agent_command]
|
|
||||||
provider_prompt_args = prompt_args(
|
provider_prompt_args = prompt_args(
|
||||||
self._agent_prompt_mode, self._prompt_path, argv=argv,
|
self._agent_prompt_mode, self._prompt_path, argv=argv,
|
||||||
)
|
)
|
||||||
@@ -148,16 +139,16 @@ class SmolmachinesBottle(Bottle):
|
|||||||
on both backends. Pass `user="root"` for tests that need
|
on both backends. Pass `user="root"` for tests that need
|
||||||
root.
|
root.
|
||||||
|
|
||||||
`runuser -u <user> -- /bin/sh -c <script>` switches UID
|
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
||||||
without invoking a login shell; HOME / USER are set via
|
without invoking a login shell, then sets HOME / USER and the
|
||||||
`smolvm -e` (see `_env_flags_for`)."""
|
bottle env in the child process."""
|
||||||
argv = (
|
argv = [
|
||||||
_env_flags_for(user)
|
"--", "runuser", "-u", user, "--",
|
||||||
+ _guest_env_flags(self._guest_env)
|
"env", *_env_assignments_for(user, self._guest_env),
|
||||||
+ ["--", "runuser", "-u", user, "--", "/bin/sh", "-c", script]
|
"/bin/sh", "-c", script,
|
||||||
)
|
]
|
||||||
# _smolvm.machine_exec expects argv (the bit after `--`);
|
# Call smolvm directly because this path needs the host-side
|
||||||
# the -e flags go before, so call smolvm directly.
|
# subprocess capture shape used by the Docker backend.
|
||||||
r = subprocess.run(
|
r = subprocess.run(
|
||||||
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
||||||
capture_output=True, text=True, check=False,
|
capture_output=True, text=True, check=False,
|
||||||
|
|||||||
@@ -8,25 +8,20 @@ in chunk 4."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import sys
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import PromptMode
|
||||||
from ...egress import EgressPlan
|
|
||||||
from ...git_gate import GitGatePlan
|
|
||||||
from ...log import info
|
|
||||||
from ...pipelock import PipelockProxyPlan
|
from ...pipelock import PipelockProxyPlan
|
||||||
from ...supervise import SupervisePlan
|
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
from ..print_util import print_multi, visible_agent_env_names
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SmolmachinesBottlePlan(BottlePlan):
|
class SmolmachinesBottlePlan(BottlePlan):
|
||||||
"""Resolved fields the launch step needs to bring up the bottle.
|
"""Resolved fields the launch step needs to bring up the bottle.
|
||||||
|
|
||||||
Inherits `spec` and `stage_dir` from BottlePlan."""
|
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
|
||||||
|
`supervise_plan`, and `agent_provision` from BottlePlan."""
|
||||||
|
|
||||||
slug: str
|
slug: str
|
||||||
# Per-bottle docker subnet for the sidecar bundle container.
|
# Per-bottle docker subnet for the sidecar bundle container.
|
||||||
@@ -68,7 +63,7 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
# empty when the agent has no prompt — claude-code reads it
|
# empty when the agent has no prompt — claude-code reads it
|
||||||
# via --append-system-prompt-file only when non-empty.
|
# via --append-system-prompt-file only when non-empty.
|
||||||
prompt_file: Path
|
prompt_file: Path
|
||||||
# Inner Plans for the four bundle daemons. The same shape the
|
# Inner Plans for the sidecar bundle daemons. The same shape the
|
||||||
# docker backend uses — same `.prepare()` calls produced
|
# docker backend uses — same `.prepare()` calls produced
|
||||||
# them — but our launch step doesn't populate the
|
# them — but our launch step doesn't populate the
|
||||||
# docker-specific network fields (internal_network,
|
# docker-specific network fields (internal_network,
|
||||||
@@ -77,11 +72,6 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||||
# at their dataclass defaults.
|
# at their dataclass defaults.
|
||||||
proxy_plan: PipelockProxyPlan
|
proxy_plan: PipelockProxyPlan
|
||||||
git_gate_plan: GitGatePlan
|
|
||||||
egress_plan: EgressPlan
|
|
||||||
# None when bottle.supervise is False, matching the docker
|
|
||||||
# backend's convention.
|
|
||||||
supervise_plan: SupervisePlan | None
|
|
||||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||||
# networking; docker container IPs live in the daemon's VM),
|
# networking; docker container IPs live in the daemon's VM),
|
||||||
@@ -93,43 +83,19 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
agent_proxy_url: str = ""
|
agent_proxy_url: str = ""
|
||||||
agent_git_gate_host: str = ""
|
agent_git_gate_host: str = ""
|
||||||
agent_supervise_url: str = ""
|
agent_supervise_url: str = ""
|
||||||
agent_command: str = "claude"
|
|
||||||
agent_prompt_mode: PromptMode = "append_file"
|
|
||||||
agent_provider_template: str = "claude"
|
|
||||||
agent_dockerfile_path: str = ""
|
|
||||||
|
|
||||||
def print(self, *, remote_control: bool) -> None:
|
@property
|
||||||
"""Compact y/N preflight. Same shape as the Docker
|
def agent_command(self) -> str:
|
||||||
backend's so operators see one format across backends."""
|
return self.agent_provision.command
|
||||||
del remote_control # not surfaced in the compact summary
|
|
||||||
spec = self.spec
|
|
||||||
manifest = spec.manifest
|
|
||||||
agent = manifest.agents[spec.agent_name]
|
|
||||||
bottle = manifest.bottle_for(spec.agent_name)
|
|
||||||
|
|
||||||
env_names = visible_agent_env_names(
|
@property
|
||||||
sorted(bottle.env.keys()),
|
def agent_prompt_mode(self) -> PromptMode:
|
||||||
agent_provider_template=self.agent_provider_template,
|
return self.agent_provision.prompt_mode
|
||||||
)
|
|
||||||
upstreams = [
|
|
||||||
f"{g.Name} → {g.Upstream}" for g in bottle.git
|
|
||||||
]
|
|
||||||
# Use the resolved egress_plan (lowercase `host` on the
|
|
||||||
# plan-level EgressRoute) rather than `bottle.egress.routes`,
|
|
||||||
# which is the manifest's capitalized-attr form.
|
|
||||||
routes = [r.host for r in self.egress_plan.routes]
|
|
||||||
|
|
||||||
print(file=sys.stderr)
|
@property
|
||||||
info(f"agent : {spec.agent_name}")
|
def agent_provider_template(self) -> str:
|
||||||
info(f"provider : {self.agent_provider_template}")
|
return self.agent_provision.template
|
||||||
print_multi("env ", env_names)
|
|
||||||
print_multi("skills ", list(agent.skills))
|
@property
|
||||||
info(f"bottle : {agent.bottle}")
|
def agent_dockerfile_path(self) -> str:
|
||||||
identity = manifest.git_identity_summary(spec.agent_name)
|
return self.agent_provision.dockerfile
|
||||||
if identity:
|
|
||||||
info(f" git identity : {identity}")
|
|
||||||
if upstreams:
|
|
||||||
print_multi(" git gate ", upstreams)
|
|
||||||
if routes:
|
|
||||||
print_multi(" egress ", routes)
|
|
||||||
print(file=sys.stderr)
|
|
||||||
|
|||||||
@@ -21,12 +21,14 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import os
|
import os
|
||||||
import time
|
|
||||||
from contextlib import ExitStack, contextmanager
|
from contextlib import ExitStack, contextmanager
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
|
|
||||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
|
from ...egress import (
|
||||||
|
EGRESS_ROUTES_IN_CONTAINER,
|
||||||
|
egress_resolve_token_values,
|
||||||
|
)
|
||||||
from ...pipelock import (
|
from ...pipelock import (
|
||||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||||
@@ -45,7 +47,6 @@ from ..docker.git_gate import (
|
|||||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||||
GIT_GATE_HOOK_IN_CONTAINER,
|
GIT_GATE_HOOK_IN_CONTAINER,
|
||||||
GIT_GATE_PORT as _GIT_GATE_PORT,
|
|
||||||
)
|
)
|
||||||
from ..docker.pipelock import (
|
from ..docker.pipelock import (
|
||||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
BUNDLE_LOCAL_PIPELOCK_URL,
|
||||||
@@ -77,6 +78,7 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
|
|||||||
# them up post-start. Pipelock's port is an env-overridable string
|
# them up post-start. Pipelock's port is an env-overridable string
|
||||||
# in docker.pipelock; coerce to int here.
|
# in docker.pipelock; coerce to int here.
|
||||||
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
|
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
|
||||||
|
_GIT_HTTP_PORT = 9420
|
||||||
_SUPERVISE_PORT = SUPERVISE_PORT
|
_SUPERVISE_PORT = SUPERVISE_PORT
|
||||||
|
|
||||||
|
|
||||||
@@ -91,199 +93,23 @@ def launch(
|
|||||||
via the ExitStack."""
|
via the ExitStack."""
|
||||||
stack = ExitStack()
|
stack = ExitStack()
|
||||||
try:
|
try:
|
||||||
# 1. Reserve a loopback alias for this bottle. macOS only
|
loopback_ip, network = _allocate_resources(plan, stack)
|
||||||
# routes 127.0.0.1 by default; the per-bottle alias is
|
plan = _mint_certs(plan)
|
||||||
# what bundles the docker port-publishes and TSI allowlist
|
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||||
# against, so this bottle can't reach other bottles' (or
|
plan = _discover_urls(plan, loopback_ip)
|
||||||
# other host services') ports on the loopback. Lazy
|
|
||||||
# sudo-driven on first use per boot. No-op on Linux.
|
|
||||||
_loopback.ensure_pool()
|
|
||||||
loopback_ip = _loopback.allocate(plan.slug)
|
|
||||||
|
|
||||||
# 2. Per-bottle docker bridge.
|
# Build the agent image and pack it into a `.smolmachine`
|
||||||
network = _bundle.bundle_network_name(plan.slug)
|
# artifact (or hit the per-Dockerfile-digest cache). Runs
|
||||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
# here, not in prepare, so the docker-build output doesn't
|
||||||
stack.callback(_bundle.remove_bundle_network, network)
|
# garble the dashboard's preflight modal.
|
||||||
|
|
||||||
# 2. Mint per-bottle CAs and update the inner Plans with
|
|
||||||
# their launch-time paths. pipelock always runs in the
|
|
||||||
# bundle; egress's CA is only minted when the bottle
|
|
||||||
# declares routes (otherwise egress runs idle without
|
|
||||||
# MITM and the CA files would be unused).
|
|
||||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
|
||||||
proxy_plan = dataclasses.replace(
|
|
||||||
plan.proxy_plan,
|
|
||||||
ca_cert_host_path=ca_cert_host,
|
|
||||||
ca_key_host_path=ca_key_host,
|
|
||||||
)
|
|
||||||
egress_plan = plan.egress_plan
|
|
||||||
if egress_plan.routes:
|
|
||||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
|
||||||
plan.egress_plan.routes_path.parent,
|
|
||||||
)
|
|
||||||
egress_plan = dataclasses.replace(
|
|
||||||
egress_plan,
|
|
||||||
mitmproxy_ca_host_path=egress_ca_host,
|
|
||||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
|
||||||
pipelock_ca_host_path=ca_cert_host,
|
|
||||||
# On smolmachines, egress's upstream is pipelock
|
|
||||||
# on the bundle's localhost — they're in the same
|
|
||||||
# container's network namespace.
|
|
||||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
|
||||||
)
|
|
||||||
plan = dataclasses.replace(
|
|
||||||
plan, proxy_plan=proxy_plan, egress_plan=egress_plan,
|
|
||||||
)
|
|
||||||
|
|
||||||
# 3. Build the BundleLaunchSpec from the (now-resolved)
|
|
||||||
# inner Plans: daemon subset, env, bind-mounts, and the
|
|
||||||
# loopback alias to bind published ports against. The
|
|
||||||
# spec's ports_to_publish list expands depending on which
|
|
||||||
# daemons the agent needs to reach from the smolvm guest.
|
|
||||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
|
||||||
token_env = _resolve_token_env(plan, os.environ)
|
|
||||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
|
||||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
|
||||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
|
||||||
|
|
||||||
# 4. Discover the host-side ports docker assigned for the
|
|
||||||
# bundle's published container ports, and bind the
|
|
||||||
# agent's URLs to `<loopback_ip>:<host port>`. Docker
|
|
||||||
# container IPs (192.168.x.x in the daemon's bridge)
|
|
||||||
# aren't reachable from the smolvm guest on macOS — TSI
|
|
||||||
# uses macOS networking, and macOS sees the daemon's
|
|
||||||
# bridge via the published-port loopback forward only.
|
|
||||||
#
|
|
||||||
# Proxy hop order matches the docker backend: when the
|
|
||||||
# bottle declares egress routes, the agent's first hop is
|
|
||||||
# egress (for token injection), then pipelock. Without
|
|
||||||
# routes, the agent dials pipelock directly. Whichever
|
|
||||||
# one is "agent-facing" is the daemon whose port we
|
|
||||||
# publish on host loopback; the other stays bundle-
|
|
||||||
# internal as the upstream proxy.
|
|
||||||
if plan.egress_plan.routes:
|
|
||||||
agent_facing_port = _EGRESS_PORT
|
|
||||||
else:
|
|
||||||
agent_facing_port = _PIPELOCK_PORT
|
|
||||||
agent_facing_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
|
||||||
agent_git_gate_host = ""
|
|
||||||
if plan.git_gate_plan.upstreams:
|
|
||||||
git_gate_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, _GIT_GATE_PORT, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
|
||||||
agent_supervise_url = ""
|
|
||||||
if plan.supervise_plan is not None:
|
|
||||||
supervise_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
|
||||||
|
|
||||||
# Stamp the URLs onto the plan + guest_env. provision_git
|
|
||||||
# and provision_supervise read the plan fields; the agent
|
|
||||||
# reads guest_env on every exec_agent.
|
|
||||||
#
|
|
||||||
# NO_PROXY has to include the per-bottle loopback alias —
|
|
||||||
# otherwise claude's HTTPS_PROXY catches direct calls to
|
|
||||||
# the supervise URL (`http://<alias>:<port>/`) and proxies
|
|
||||||
# them through egress, which has no route for the alias
|
|
||||||
# and rejects with "Failed to connect". The git-gate URL
|
|
||||||
# uses git://, not affected by HTTP_PROXY, so the alias
|
|
||||||
# only has to be in NO_PROXY for the MCP / supervise
|
|
||||||
# path. Append rather than overwrite so prepare.py's
|
|
||||||
# `localhost,127.0.0.1` baseline stays in place.
|
|
||||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
|
||||||
guest_env = {
|
|
||||||
**plan.guest_env,
|
|
||||||
"HTTPS_PROXY": agent_proxy_url,
|
|
||||||
"HTTP_PROXY": agent_proxy_url,
|
|
||||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
|
||||||
}
|
|
||||||
if agent_git_gate_host:
|
|
||||||
guest_env["GIT_GATE_URL"] = f"git://{agent_git_gate_host}"
|
|
||||||
if agent_supervise_url:
|
|
||||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
|
||||||
plan = dataclasses.replace(
|
|
||||||
plan,
|
|
||||||
guest_env=guest_env,
|
|
||||||
agent_proxy_url=agent_proxy_url,
|
|
||||||
agent_git_gate_host=agent_git_gate_host,
|
|
||||||
agent_supervise_url=agent_supervise_url,
|
|
||||||
)
|
|
||||||
|
|
||||||
# 5. Build the agent image and pack it into a
|
|
||||||
# `.smolmachine` artifact (or hit the per-Dockerfile-digest
|
|
||||||
# cache). Runs here, not in prepare, so the docker-build
|
|
||||||
# output doesn't garble the dashboard's preflight modal:
|
|
||||||
# both the curses-endwin path and the tmux pane-routing
|
|
||||||
# path redirect stderr around `launch` already.
|
|
||||||
agent_from_path = _ensure_smolmachine(
|
agent_from_path = _ensure_smolmachine(
|
||||||
plan.agent_image_ref,
|
plan.agent_image_ref,
|
||||||
dockerfile=plan.agent_dockerfile_path,
|
dockerfile=plan.agent_dockerfile_path,
|
||||||
)
|
)
|
||||||
|
|
||||||
# smolvm VM. --from carries the pre-packed .smolmachine
|
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||||
# artifact; --allow-cidr + -e carry the per-bottle TSI
|
_init_vm(plan)
|
||||||
# allowlist + env. The allowlist is the per-bottle
|
|
||||||
# loopback alias — narrowing it to one /32 keeps the
|
|
||||||
# agent from reaching other host loopback services or
|
|
||||||
# other bottles' published ports. Smolfile isn't usable
|
|
||||||
# here — smolvm 0.8.0 makes `--from` and `--smolfile`
|
|
||||||
# mutually exclusive.
|
|
||||||
_smolvm.machine_create(
|
|
||||||
plan.machine_name,
|
|
||||||
from_path=agent_from_path,
|
|
||||||
allow_cidrs=[f"{loopback_ip}/32"],
|
|
||||||
env=plan.guest_env,
|
|
||||||
)
|
|
||||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
|
||||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently
|
|
||||||
# dropped when combined with `--from`. Patch the persisted
|
|
||||||
# state DB to set the allowlist before start so the booted
|
|
||||||
# VM's TSI actually enforces. See loopback_alias's module
|
|
||||||
# docstring for the investigation that led here.
|
|
||||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
|
||||||
_smolvm.machine_start(plan.machine_name)
|
|
||||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
|
||||||
|
|
||||||
# 6. Repair filesystem ownership + perms that smolvm's
|
|
||||||
# pack process remapped to the host invoker's uid (501
|
|
||||||
# on macOS) rather than preserving the image's expected
|
|
||||||
# ownership.
|
|
||||||
#
|
|
||||||
# - /home/node → node:node so the node user can write
|
|
||||||
# its own dotfiles (claude appendFileSync on
|
|
||||||
# ~/.claude.json otherwise bails with ENOENT/EPERM
|
|
||||||
# and the TUI hangs without surfacing the error).
|
|
||||||
# - /tmp + /var/tmp → root:root mode 1777 so non-root
|
|
||||||
# processes can create their per-uid scratch dirs
|
|
||||||
# (claude-code creates /tmp/claude-<uid>/ as soon as
|
|
||||||
# it spawns a Bash tool call).
|
|
||||||
#
|
|
||||||
# All folded into one sh -c so we only pay one
|
|
||||||
# machine_exec round trip — back-to-back exec calls
|
|
||||||
# right after machine_start hit a SIGKILL race in
|
|
||||||
# libkrun's exec channel (see provision_ca for the
|
|
||||||
# other half of this same workaround).
|
|
||||||
_smolvm.machine_exec(plan.machine_name, [
|
|
||||||
"sh", "-c",
|
|
||||||
"chown -R node:node /home/node && "
|
|
||||||
"chown root:root /tmp /var/tmp && "
|
|
||||||
"chmod 1777 /tmp /var/tmp",
|
|
||||||
])
|
|
||||||
|
|
||||||
# Wait briefly for the VM to settle. Back-to-back smolvm
|
|
||||||
# machine_exec calls immediately after machine_start
|
|
||||||
# occasionally SIGKILL the in-VM child at ~100ms (looks
|
|
||||||
# like a VM warm-up race in libkrun's exec channel).
|
|
||||||
# 1.5s is empirically enough to dodge it; provisioning
|
|
||||||
# already takes seconds so the wait is amortized.
|
|
||||||
time.sleep(1.5)
|
|
||||||
|
|
||||||
# 7. Provision (CA / prompt / skills / git / supervise).
|
|
||||||
prompt_path = provision(plan, plan.machine_name)
|
prompt_path = provision(plan, plan.machine_name)
|
||||||
|
|
||||||
yield SmolmachinesBottle(
|
yield SmolmachinesBottle(
|
||||||
@@ -297,6 +123,180 @@ def launch(
|
|||||||
stack.close()
|
stack.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _allocate_resources(
|
||||||
|
plan: SmolmachinesBottlePlan,
|
||||||
|
stack: ExitStack,
|
||||||
|
) -> tuple[str, str]:
|
||||||
|
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||||
|
|
||||||
|
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
||||||
|
scopes TSI's allowlist to this bottle's published ports so the
|
||||||
|
agent can't reach other bottles' or host services' ports on
|
||||||
|
loopback. No-op on Linux."""
|
||||||
|
_loopback.ensure_pool()
|
||||||
|
loopback_ip = _loopback.allocate(plan.slug)
|
||||||
|
network = _bundle.bundle_network_name(plan.slug)
|
||||||
|
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||||
|
stack.callback(_bundle.remove_bundle_network, network)
|
||||||
|
return loopback_ip, network
|
||||||
|
|
||||||
|
|
||||||
|
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||||
|
"""Mint per-bottle CAs and return the plan with CA paths filled.
|
||||||
|
|
||||||
|
Pipelock always runs in the bundle. Egress's CA is only minted
|
||||||
|
when the bottle declares routes — otherwise egress runs idle
|
||||||
|
without MITM and the CA files would be unused."""
|
||||||
|
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
||||||
|
proxy_plan = dataclasses.replace(
|
||||||
|
plan.proxy_plan,
|
||||||
|
ca_cert_host_path=ca_cert_host,
|
||||||
|
ca_key_host_path=ca_key_host,
|
||||||
|
)
|
||||||
|
egress_plan = plan.egress_plan
|
||||||
|
if egress_plan.routes:
|
||||||
|
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||||
|
plan.egress_plan.routes_path.parent,
|
||||||
|
)
|
||||||
|
egress_plan = dataclasses.replace(
|
||||||
|
egress_plan,
|
||||||
|
mitmproxy_ca_host_path=egress_ca_host,
|
||||||
|
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||||
|
pipelock_ca_host_path=ca_cert_host,
|
||||||
|
# On smolmachines, egress's upstream is pipelock on the
|
||||||
|
# bundle's localhost — they're in the same container's
|
||||||
|
# network namespace.
|
||||||
|
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||||
|
)
|
||||||
|
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
|
||||||
|
|
||||||
|
|
||||||
|
def _start_bundle(
|
||||||
|
plan: SmolmachinesBottlePlan,
|
||||||
|
network: str,
|
||||||
|
loopback_ip: str,
|
||||||
|
stack: ExitStack,
|
||||||
|
) -> SmolmachinesBottlePlan:
|
||||||
|
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||||
|
sidecar bundle container, and register teardown."""
|
||||||
|
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||||
|
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||||
|
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||||
|
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||||
|
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||||
|
return plan
|
||||||
|
|
||||||
|
|
||||||
|
def _discover_urls(
|
||||||
|
plan: SmolmachinesBottlePlan,
|
||||||
|
loopback_ip: str,
|
||||||
|
) -> SmolmachinesBottlePlan:
|
||||||
|
"""Discover host-side ports for published container ports and
|
||||||
|
return the plan with URLs + guest_env stamped in.
|
||||||
|
|
||||||
|
Docker container IPs (192.168.x.x in the daemon's bridge)
|
||||||
|
aren't reachable from the smolvm guest on macOS — TSI uses
|
||||||
|
macOS networking, and macOS sees the daemon's bridge via the
|
||||||
|
published-port loopback forward only.
|
||||||
|
|
||||||
|
Proxy hop order: when the bottle declares egress routes, the
|
||||||
|
agent's first hop is egress (for token injection), then
|
||||||
|
pipelock. Without routes, the agent dials pipelock directly.
|
||||||
|
NO_PROXY includes the per-bottle loopback alias so the
|
||||||
|
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
||||||
|
if plan.egress_plan.routes:
|
||||||
|
agent_facing_port = _EGRESS_PORT
|
||||||
|
else:
|
||||||
|
agent_facing_port = _PIPELOCK_PORT
|
||||||
|
agent_facing_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||||
|
|
||||||
|
agent_git_gate_host = ""
|
||||||
|
if plan.git_gate_plan.upstreams:
|
||||||
|
git_gate_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||||
|
|
||||||
|
agent_supervise_url = ""
|
||||||
|
if plan.supervise_plan is not None:
|
||||||
|
supervise_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||||
|
|
||||||
|
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||||
|
guest_env = {
|
||||||
|
**plan.guest_env,
|
||||||
|
"HTTPS_PROXY": agent_proxy_url,
|
||||||
|
"HTTP_PROXY": agent_proxy_url,
|
||||||
|
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||||
|
}
|
||||||
|
if agent_git_gate_host:
|
||||||
|
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||||
|
if agent_supervise_url:
|
||||||
|
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||||
|
|
||||||
|
return dataclasses.replace(
|
||||||
|
plan,
|
||||||
|
guest_env=guest_env,
|
||||||
|
agent_proxy_url=agent_proxy_url,
|
||||||
|
agent_git_gate_host=agent_git_gate_host,
|
||||||
|
agent_supervise_url=agent_supervise_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _launch_vm(
|
||||||
|
plan: SmolmachinesBottlePlan,
|
||||||
|
agent_from_path: Path,
|
||||||
|
loopback_ip: str,
|
||||||
|
stack: ExitStack,
|
||||||
|
) -> None:
|
||||||
|
"""Create, patch, and start the smolvm VM; register teardown.
|
||||||
|
|
||||||
|
--allow-cidr is the per-bottle loopback alias so the guest can
|
||||||
|
only reach this bottle's bundle ports. force_allowlist patches
|
||||||
|
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
||||||
|
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||||
|
and --smolfile mutually exclusive."""
|
||||||
|
_smolvm.machine_create(
|
||||||
|
plan.machine_name,
|
||||||
|
from_path=agent_from_path,
|
||||||
|
allow_cidrs=[f"{loopback_ip}/32"],
|
||||||
|
env=plan.guest_env,
|
||||||
|
)
|
||||||
|
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||||
|
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
||||||
|
# when combined with `--from`. Patch the persisted state DB
|
||||||
|
# before start so the booted VM's TSI actually enforces.
|
||||||
|
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||||
|
_smolvm.machine_start(plan.machine_name)
|
||||||
|
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||||
|
|
||||||
|
|
||||||
|
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||||
|
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||||
|
|
||||||
|
Ownership repair: smolvm's pack process remaps files to the host
|
||||||
|
invoker's uid (501 on macOS). /home/node must be node:node so
|
||||||
|
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||||
|
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||||
|
All folded into one sh -c to avoid back-to-back exec calls
|
||||||
|
immediately after machine_start (libkrun exec-channel race).
|
||||||
|
|
||||||
|
wait_exec_ready polls until the exec channel is ready for the
|
||||||
|
subsequent provision calls, replacing the empirical sleep."""
|
||||||
|
_smolvm.machine_exec(plan.machine_name, [
|
||||||
|
"sh", "-c",
|
||||||
|
"chown -R node:node /home/node && "
|
||||||
|
"chown root:root /tmp /var/tmp && "
|
||||||
|
"chmod 1777 /tmp /var/tmp",
|
||||||
|
])
|
||||||
|
_smolvm.wait_exec_ready(plan.machine_name)
|
||||||
|
|
||||||
|
|
||||||
def _bundle_launch_spec(
|
def _bundle_launch_spec(
|
||||||
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str,
|
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str,
|
||||||
) -> _bundle.BundleLaunchSpec:
|
) -> _bundle.BundleLaunchSpec:
|
||||||
@@ -305,10 +305,10 @@ def _bundle_launch_spec(
|
|||||||
Daemons in the CSV:
|
Daemons in the CSV:
|
||||||
- egress + pipelock are always present (pipelock is the
|
- egress + pipelock are always present (pipelock is the
|
||||||
agent's first hop; egress is its upstream).
|
agent's first hop; egress is its upstream).
|
||||||
- git-gate is conditional on plan.git_gate_plan.upstreams.
|
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
||||||
- supervise is conditional on plan.supervise_plan.
|
- supervise is conditional on plan.supervise_plan.
|
||||||
|
|
||||||
Env + volumes are the union of the four daemons' needs, with
|
Env + volumes are the union of the sidecar daemons' needs, with
|
||||||
daemon-private values only (HTTPS_PROXY is scoped to the
|
daemon-private values only (HTTPS_PROXY is scoped to the
|
||||||
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
||||||
bind-address PR)."""
|
bind-address PR)."""
|
||||||
@@ -320,10 +320,9 @@ def _bundle_launch_spec(
|
|||||||
# is "agent-facing" gets its port published on the host
|
# is "agent-facing" gets its port published on the host
|
||||||
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
||||||
# other stays bundle-internal. The bundle is NOT reachable by
|
# other stays bundle-internal. The bundle is NOT reachable by
|
||||||
# bridge IP from the smolvm guest, so the
|
# bridge IP from the smolvm guest on macOS — TSI uses macOS
|
||||||
# PRD-0023-chunk-3 EGRESS_LISTEN_HOST=127.0.0.1 mitigation
|
# networking, and macOS sees the daemon's bridge via the
|
||||||
# isn't needed: the agent can only dial whatever daemon's
|
# published-port loopback forward only.
|
||||||
# host port we publish, period.
|
|
||||||
|
|
||||||
# --- pipelock ---------------------------------------------
|
# --- pipelock ---------------------------------------------
|
||||||
pp = plan.proxy_plan
|
pp = plan.proxy_plan
|
||||||
@@ -350,10 +349,9 @@ def _bundle_launch_spec(
|
|||||||
env.append(token_env)
|
env.append(token_env)
|
||||||
|
|
||||||
# --- git-gate ---------------------------------------------
|
# --- git-gate ---------------------------------------------
|
||||||
extra_hosts: list[str] = []
|
|
||||||
gp = plan.git_gate_plan
|
gp = plan.git_gate_plan
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
daemons.append("git-gate")
|
daemons += ["git-gate", "git-http"]
|
||||||
volumes += [
|
volumes += [
|
||||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
|
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
|
||||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
|
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
|
||||||
@@ -395,7 +393,7 @@ def _bundle_launch_spec(
|
|||||||
else:
|
else:
|
||||||
ports_to_publish = [_PIPELOCK_PORT]
|
ports_to_publish = [_PIPELOCK_PORT]
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
ports_to_publish.append(_GIT_GATE_PORT)
|
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||||
if sp is not None:
|
if sp is not None:
|
||||||
ports_to_publish.append(_SUPERVISE_PORT)
|
ports_to_publish.append(_SUPERVISE_PORT)
|
||||||
|
|
||||||
@@ -414,15 +412,13 @@ def _bundle_launch_spec(
|
|||||||
|
|
||||||
|
|
||||||
def _resolve_token_env(
|
def _resolve_token_env(
|
||||||
plan: SmolmachinesBottlePlan, host_env: object
|
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Resolve the egress token env-var values from the host's
|
"""Resolve the egress token env-var values from the host's
|
||||||
environ so they reach the bundle's process env via docker's
|
environ so they reach the bundle's process env via docker's
|
||||||
`-e NAME` inheritance. Empty when no routes declare auth."""
|
`-e NAME` inheritance. Empty when no routes declare auth."""
|
||||||
ep = plan.egress_plan
|
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
|
||||||
if not ep.routes:
|
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||||
return {}
|
|
||||||
return egress_resolve_token_values(ep.token_env_map, dict(host_env))
|
|
||||||
|
|
||||||
|
|
||||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||||
|
|||||||
@@ -45,6 +45,7 @@ alias gets handed to a new bottle."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import fcntl
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import platform
|
import platform
|
||||||
@@ -83,6 +84,14 @@ _POOL_START = 16
|
|||||||
_POOL_END = 31 # inclusive
|
_POOL_END = 31 # inclusive
|
||||||
|
|
||||||
|
|
||||||
|
# File lock that serialises concurrent allocate() calls so two
|
||||||
|
# simultaneous launches can't read the same docker state and claim
|
||||||
|
# the same alias. Narrowed to the allocate() call itself; docker run
|
||||||
|
# runs after the lock is released. Once the container is running it
|
||||||
|
# appears in docker state and future allocate() calls will see it.
|
||||||
|
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
||||||
|
|
||||||
|
|
||||||
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
||||||
def _pool_addresses() -> list[str]:
|
def _pool_addresses() -> list[str]:
|
||||||
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
||||||
@@ -179,9 +188,20 @@ def allocate(slug: str) -> str:
|
|||||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
||||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
`127.0.0.1` is fine to share and we skip the alias dance.
|
||||||
This still returns a deterministic address so launch.py's
|
This still returns a deterministic address so launch.py's
|
||||||
callers don't have to branch on platform."""
|
callers don't have to branch on platform.
|
||||||
|
|
||||||
|
An exclusive file lock serialises concurrent calls so two
|
||||||
|
simultaneous launches don't read the same docker state and
|
||||||
|
claim the same alias."""
|
||||||
if not _is_macos():
|
if not _is_macos():
|
||||||
return "127.0.0.1"
|
return "127.0.0.1"
|
||||||
|
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
||||||
|
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||||
|
return _allocate_locked()
|
||||||
|
|
||||||
|
|
||||||
|
def _allocate_locked() -> str:
|
||||||
in_use = _aliases_in_use()
|
in_use = _aliases_in_use()
|
||||||
for ip in _pool_addresses():
|
for ip in _pool_addresses():
|
||||||
if ip not in in_use:
|
if ip not in in_use:
|
||||||
|
|||||||
@@ -12,9 +12,10 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
|
from dataclasses import replace
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import runtime_for
|
from ...agent_provider import agent_provision_plan, runtime_for
|
||||||
from ...backend import BottleSpec
|
from ...backend import BottleSpec
|
||||||
from ...backend.docker.bottle_state import (
|
from ...backend.docker.bottle_state import (
|
||||||
BottleMetadata,
|
BottleMetadata,
|
||||||
@@ -27,9 +28,11 @@ from ...backend.docker.bottle_state import (
|
|||||||
write_metadata,
|
write_metadata,
|
||||||
)
|
)
|
||||||
from ...egress import Egress
|
from ...egress import Egress
|
||||||
|
from ...env import resolve_env
|
||||||
from ...git_gate import GitGate
|
from ...git_gate import GitGate
|
||||||
from ...pipelock import PipelockProxy
|
from ...pipelock import PipelockProxy
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
|
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||||
from .bottle_plan import SmolmachinesBottlePlan
|
from .bottle_plan import SmolmachinesBottlePlan
|
||||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||||
|
|
||||||
@@ -58,6 +61,8 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
|
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", "/home/node")
|
||||||
|
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||||
|
|
||||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||||
|
|
||||||
@@ -69,72 +74,34 @@ def resolve_plan(
|
|||||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||||
copy_cwd=spec.copy_cwd,
|
copy_cwd=spec.copy_cwd,
|
||||||
started_at=datetime.now(timezone.utc).isoformat(),
|
started_at=datetime.now(timezone.utc).isoformat(),
|
||||||
# No compose project for smolmachines bottles; chunk 4
|
|
||||||
# will give dashboard discovery a backend-specific path.
|
|
||||||
compose_project="",
|
compose_project="",
|
||||||
|
backend="smolmachines",
|
||||||
))
|
))
|
||||||
|
|
||||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||||
|
|
||||||
# Agent's env: the prepare-time view doesn't yet know the
|
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||||
# host loopback ports the bundle's daemons get published on
|
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||||
# (those come from docker AFTER `docker run` returns), so
|
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are
|
# values still reach the guest as -e K=V smolvm flags because
|
||||||
# populated in launch.py and stamped onto guest_env there.
|
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||||
# What we set here is the part that doesn't depend on
|
# the known argv-exposure gap documented in PRD 0038.
|
||||||
# bundle bringup — bottle.env literals, the empty-NO_PROXY
|
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||||
# safe default, and the TLS trust env trio
|
# in launch.py after bundle bringup.
|
||||||
# (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE)
|
resolved = resolve_env(manifest, spec.agent_name)
|
||||||
# pointing at Debian's update-ca-certificates output bundle.
|
|
||||||
guest_env: dict[str, str] = {
|
guest_env: dict[str, str] = {
|
||||||
**bottle.env,
|
**resolved.literals,
|
||||||
|
**resolved.forwarded,
|
||||||
"NO_PROXY": "localhost,127.0.0.1",
|
"NO_PROXY": "localhost,127.0.0.1",
|
||||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
}
|
}
|
||||||
|
|
||||||
# Inner Plans for the four bundle daemons. The ABCs are
|
|
||||||
# platform-neutral — `.prepare()` writes config files + returns
|
|
||||||
# a Plan dataclass with no backend-specific assumptions. State
|
|
||||||
# dirs are still keyed by slug under the docker backend's
|
|
||||||
# bottle_state layout (shared on-host convention; not a docker
|
|
||||||
# dependency).
|
|
||||||
pipelock_dir = pipelock_state_dir(slug)
|
|
||||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
proxy_plan = PipelockProxy().prepare(bottle, slug, pipelock_dir)
|
|
||||||
|
|
||||||
git_gate_dir = git_gate_state_dir(slug)
|
git_gate_dir = git_gate_state_dir(slug)
|
||||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||||
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
|
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
|
||||||
|
|
||||||
egress_dir = egress_state_dir(slug)
|
|
||||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
egress_plan = Egress().prepare(bottle, slug, egress_dir)
|
|
||||||
|
|
||||||
# Claude-code refuses to start without *something* it
|
|
||||||
# recognises as a credential. When the bottle has an egress
|
|
||||||
# route carrying the `claude_code_oauth` role marker, egress
|
|
||||||
# strips + re-injects the real Authorization header on the
|
|
||||||
# outbound leg using a token held in egress's own environ — so
|
|
||||||
# the agent gets a non-secret placeholder here (matches the
|
|
||||||
# docker backend's forwarded_env logic in
|
|
||||||
# bot_bottle/backend/docker/prepare.py).
|
|
||||||
has_provider_auth = any(
|
|
||||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
|
||||||
)
|
|
||||||
if has_provider_auth:
|
|
||||||
guest_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
|
||||||
if provider.template == "claude" and has_provider_auth:
|
|
||||||
guest_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
|
||||||
guest_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
|
||||||
|
|
||||||
supervise_plan = None
|
|
||||||
if bottle.supervise:
|
|
||||||
supervise_dir = supervise_state_dir(slug)
|
|
||||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
|
||||||
|
|
||||||
# Prompt file is always written (mode 0o600) so the in-VM
|
# Prompt file is always written (mode 0o600) so the in-VM
|
||||||
# path always exists. Content is the agent's `prompt`
|
# path always exists. Content is the agent's `prompt`
|
||||||
# field (markdown body) — empty for agents with no prompt.
|
# field (markdown body) — empty for agents with no prompt.
|
||||||
@@ -162,6 +129,45 @@ def resolve_plan(
|
|||||||
else:
|
else:
|
||||||
image_default = provider_runtime.image
|
image_default = provider_runtime.image
|
||||||
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||||
|
agent_provision = agent_provision_plan(
|
||||||
|
template=provider.template,
|
||||||
|
dockerfile=agent_dockerfile_path,
|
||||||
|
state_dir=agent_dir,
|
||||||
|
guest_home=guest_home,
|
||||||
|
guest_env=guest_env,
|
||||||
|
forward_host_credentials=provider.forward_host_credentials,
|
||||||
|
auth_token=provider.auth_token,
|
||||||
|
host_env=dict(os.environ),
|
||||||
|
trusted_project_path=workspace_plan.workdir,
|
||||||
|
)
|
||||||
|
merged_guest_env = dict(agent_provision.guest_env)
|
||||||
|
for key, val in agent_provision.env_vars.items():
|
||||||
|
merged_guest_env.setdefault(key, val)
|
||||||
|
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
|
||||||
|
|
||||||
|
# Inner Plans for the four bundle daemons. The ABCs are
|
||||||
|
# platform-neutral — `.prepare()` writes config files + returns
|
||||||
|
# a Plan dataclass with no backend-specific assumptions. State
|
||||||
|
# dirs are still keyed by slug under the docker backend's
|
||||||
|
# bottle_state layout (shared on-host convention; not a docker
|
||||||
|
# dependency).
|
||||||
|
pipelock_dir = pipelock_state_dir(slug)
|
||||||
|
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
proxy_plan = PipelockProxy().prepare(
|
||||||
|
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
||||||
|
)
|
||||||
|
|
||||||
|
egress_dir = egress_state_dir(slug)
|
||||||
|
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
egress_plan = Egress().prepare(
|
||||||
|
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||||
|
)
|
||||||
|
|
||||||
|
supervise_plan = None
|
||||||
|
if bottle.supervise:
|
||||||
|
supervise_dir = supervise_state_dir(slug)
|
||||||
|
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
||||||
|
|
||||||
return SmolmachinesBottlePlan(
|
return SmolmachinesBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
@@ -172,16 +178,14 @@ def resolve_plan(
|
|||||||
bundle_ip=bundle_ip,
|
bundle_ip=bundle_ip,
|
||||||
machine_name=machine_name,
|
machine_name=machine_name,
|
||||||
agent_image_ref=agent_image_ref,
|
agent_image_ref=agent_image_ref,
|
||||||
guest_env=guest_env,
|
guest_env=agent_provision.guest_env,
|
||||||
prompt_file=prompt_file,
|
prompt_file=prompt_file,
|
||||||
proxy_plan=proxy_plan,
|
proxy_plan=proxy_plan,
|
||||||
git_gate_plan=git_gate_plan,
|
git_gate_plan=git_gate_plan,
|
||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
agent_command=provider_runtime.command,
|
agent_provision=agent_provision,
|
||||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
workspace_plan=workspace_plan,
|
||||||
agent_provider_template=provider.template,
|
|
||||||
agent_dockerfile_path=agent_dockerfile_path,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ flag exists; the VM init is root), so we don't need the explicit
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
|
||||||
from ....log import die
|
from ....log import die
|
||||||
from ...util import (
|
from ...util import (
|
||||||
AGENT_CA_BUNDLE,
|
AGENT_CA_BUNDLE,
|
||||||
@@ -26,6 +28,9 @@ from .. import smolvm as _smolvm
|
|||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
_SIGKILL_EXIT = 128 + 9
|
||||||
|
|
||||||
|
|
||||||
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
"""Copy the agent-facing CA cert into the guest, rebuild the
|
"""Copy the agent-facing CA cert into the guest, rebuild the
|
||||||
trust bundle, emit a one-line fingerprint log. Called from
|
trust bundle, emit a one-line fingerprint log. Called from
|
||||||
@@ -40,17 +45,16 @@ def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|||||||
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
||||||
# `requests` / libraries that don't load the system bundle.
|
# `requests` / libraries that don't load the system bundle.
|
||||||
#
|
#
|
||||||
# chown + chmod + update-ca-certificates run in one
|
r = _install_ca(target)
|
||||||
# `sh -c` so we only pay one machine_exec round trip; the
|
if r.returncode == _SIGKILL_EXIT:
|
||||||
# `&&` chaining surfaces the first failure as the return
|
# smolvm/libkrun can SIGKILL an otherwise-normal exec
|
||||||
# code.
|
# during early-VM provisioning. `update-ca-certificates`
|
||||||
r = _smolvm.machine_exec(target, [
|
# is idempotent, so retry the same install once after a
|
||||||
"sh", "-c",
|
# short settle delay before treating it as fatal.
|
||||||
f"chown root:root {AGENT_CA_PATH} && "
|
time.sleep(1.0)
|
||||||
f"chmod 644 {AGENT_CA_PATH} && "
|
r = _install_ca(target)
|
||||||
f"update-ca-certificates",
|
|
||||||
])
|
if r.returncode != 0:
|
||||||
if r.returncode != 0 or "1 added" not in (r.stdout or ""):
|
|
||||||
# update-ca-certificates not adding our cert is fatal —
|
# update-ca-certificates not adding our cert is fatal —
|
||||||
# claude-code's TLS handshake against the egress-MITM'd
|
# claude-code's TLS handshake against the egress-MITM'd
|
||||||
# api.anthropic.com would fail downstream. Bail early
|
# api.anthropic.com would fail downstream. Bail early
|
||||||
@@ -66,6 +70,23 @@ def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|||||||
log_ca_fingerprint(cert_host_path, label)
|
log_ca_fingerprint(cert_host_path, label)
|
||||||
|
|
||||||
|
|
||||||
|
def _install_ca(target: str) -> _smolvm.SmolvmRunResult:
|
||||||
|
# chown + chmod + update-ca-certificates + bundle
|
||||||
|
# verification run in one `sh -c` so we only pay one
|
||||||
|
# machine_exec round trip; the `&&` chaining surfaces the
|
||||||
|
# first failure as the return code. The verify check is more
|
||||||
|
# stable than requiring "1 added" in stdout: a retry after a
|
||||||
|
# partially-completed first run may legitimately report "0
|
||||||
|
# added" while the cert is already installed.
|
||||||
|
return _smolvm.machine_exec(target, [
|
||||||
|
"sh", "-c",
|
||||||
|
f"chown root:root {AGENT_CA_PATH} && "
|
||||||
|
f"chmod 644 {AGENT_CA_PATH} && "
|
||||||
|
f"update-ca-certificates && "
|
||||||
|
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
# Re-exported for the launch/provision_ca caller + tests. The path
|
# Re-exported for the launch/provision_ca caller + tests. The path
|
||||||
# constants live in the shared `backend.util` (Debian's
|
# constants live in the shared `backend.util` (Debian's
|
||||||
# `update-ca-certificates` layout is the same in both backends).
|
# `update-ca-certificates` layout is the same in both backends).
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
Three concerns, all about git in the agent:
|
Three concerns, all about git in the agent:
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that
|
1. If --cwd was passed AND the host cwd has a .git, copy that
|
||||||
.git into /home/node/workspace/.git so the agent operates on
|
.git into the planned guest workspace so the agent operates on
|
||||||
the user's repo.
|
the user's repo.
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
~/.gitconfig with insteadOf rules so every git operation
|
||||||
@@ -18,7 +18,7 @@ Three concerns, all about git in the agent:
|
|||||||
Differs from `backend.docker.provision.git` in one address detail:
|
Differs from `backend.docker.provision.git` in one address detail:
|
||||||
the TSI-allowlisted guest can only reach the bundle's pinned IP
|
the TSI-allowlisted guest can only reach the bundle's pinned IP
|
||||||
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
|
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
|
||||||
are `git://<bundle_ip>:<port>/<name>.git` rather than the
|
are `http://<bundle_ip>:<port>/<name>.git` rather than the
|
||||||
docker backend's `git://git-gate/<name>.git`. The render itself
|
docker backend's `git://git-gate/<name>.git`. The render itself
|
||||||
is the shared `git_gate_render_gitconfig` on the platform-neutral
|
is the shared `git_gate_render_gitconfig` on the platform-neutral
|
||||||
git_gate module."""
|
git_gate module."""
|
||||||
@@ -58,20 +58,22 @@ def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||||
it into <guest_home>/workspace/.git and fix ownership. No-op
|
it into <guest_home>/workspace/.git and fix ownership. No-op
|
||||||
otherwise."""
|
otherwise."""
|
||||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
workspace = plan.workspace_plan
|
||||||
|
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
||||||
return
|
return
|
||||||
guest_workspace_git = f"{_guest_home()}/workspace/.git"
|
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||||
info(f"copying {plan.spec.user_cwd}/.git -> {target}:{guest_workspace_git}")
|
host_git = str(workspace.host_path / ".git")
|
||||||
|
info(f"copying {host_git} -> {target}:{guest_workspace_git}")
|
||||||
# mkdir -p the workspace dir so `machine cp` lands the .git
|
# mkdir -p the workspace dir so `machine cp` lands the .git
|
||||||
# directly there even on first-time bottles.
|
# directly there even on first-time bottles.
|
||||||
_smolvm.machine_exec(target, ["mkdir", "-p", f"{_guest_home()}/workspace"])
|
_smolvm.machine_exec(target, ["mkdir", "-p", workspace.guest_path])
|
||||||
_smolvm.machine_cp(
|
_smolvm.machine_cp(
|
||||||
f"{plan.spec.user_cwd}/.git", f"{target}:{guest_workspace_git}",
|
host_git, f"{target}:{guest_workspace_git}",
|
||||||
)
|
)
|
||||||
# `machine cp` lands files as root; the agent runs as node so
|
# `machine cp` lands files as root; the agent runs as node so
|
||||||
# the workspace tree must be chowned over.
|
# the workspace tree must be chowned over.
|
||||||
_smolvm.machine_exec(
|
_smolvm.machine_exec(
|
||||||
target, ["chown", "-R", "node:node", guest_workspace_git],
|
target, ["chown", "-R", workspace.owner, guest_workspace_git],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -82,12 +84,14 @@ def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> Non
|
|||||||
if not bottle.git:
|
if not bottle.git:
|
||||||
return
|
return
|
||||||
|
|
||||||
# `127.0.0.1:<host port>` form: the bundle's git-gate port
|
# `<loopback alias>:<host port>` form: the bundle's git-gate
|
||||||
# is published on host loopback at launch time so the
|
# HTTP port is published on host loopback at launch time so
|
||||||
# smolvm guest (which can only reach macOS networking via
|
# the smolvm guest (which can only reach macOS networking via
|
||||||
# TSI, not the docker bridge IP) can dial it. launch.py
|
# TSI, not the docker bridge IP) can dial it. launch.py
|
||||||
# populates `plan.agent_git_gate_host` after bundle bringup.
|
# populates `plan.agent_git_gate_host` after bundle bringup.
|
||||||
content = git_gate_render_gitconfig(bottle.git, plan.agent_git_gate_host)
|
content = git_gate_render_gitconfig(
|
||||||
|
bottle.git, plan.agent_git_gate_host, scheme="http",
|
||||||
|
)
|
||||||
|
|
||||||
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
||||||
# Stage the file under the plan's stage_dir so `machine cp`
|
# Stage the file under the plan's stage_dir so `machine cp`
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
"""Provision non-secret provider auth markers into a smolmachines bottle."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from ....log import die
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_provider_auth(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
"""Apply provider-owned guest setup through smolvm primitives."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
_exec(target, ["mkdir", "-p", d.guest_path], f"could not create {d.guest_path}")
|
||||||
|
_exec(target, ["chown", d.owner, d.guest_path], f"could not chown {d.guest_path}")
|
||||||
|
_exec(target, ["chmod", d.mode, d.guest_path], f"could not chmod {d.guest_path}")
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(target, list(command.argv), command.error)
|
||||||
|
for f in provision.files:
|
||||||
|
_smolvm.machine_cp(str(f.host_path), f"{target}:{f.guest_path}")
|
||||||
|
_exec(target, ["chown", f.owner, f.guest_path], f"could not chown {f.guest_path}")
|
||||||
|
_exec(target, ["chmod", f.mode, f.guest_path], f"could not chmod {f.guest_path}")
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(target, list(command.argv), command.error)
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(target: str, argv: list[str], error: str) -> None:
|
||||||
|
result = _smolvm.machine_exec(target, argv)
|
||||||
|
if result.returncode != 0:
|
||||||
|
detail = (result.stderr or result.stdout).strip()
|
||||||
|
if detail:
|
||||||
|
detail = f": {detail}"
|
||||||
|
die(f"agent provider provisioning: {error}{detail}")
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
"""Copy the operator workspace into a smolmachines guest."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shlex
|
||||||
|
|
||||||
|
from ....log import info
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_workspace(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
"""Copy host cwd contents to the planned guest workspace."""
|
||||||
|
workspace = plan.workspace_plan
|
||||||
|
if not (workspace.enabled and workspace.copy_contents):
|
||||||
|
return
|
||||||
|
|
||||||
|
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
||||||
|
guest_path_q = shlex.quote(workspace.guest_path)
|
||||||
|
guest_parent_q = shlex.quote(guest_parent)
|
||||||
|
owner_q = shlex.quote(workspace.owner)
|
||||||
|
mode_q = shlex.quote(workspace.mode)
|
||||||
|
info(f"copying {workspace.host_path} -> {target}:{workspace.guest_path}")
|
||||||
|
_smolvm.machine_exec(
|
||||||
|
target,
|
||||||
|
["sh", "-c", f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}"],
|
||||||
|
)
|
||||||
|
_smolvm.machine_cp(str(workspace.host_path), f"{target}:{workspace.guest_path}")
|
||||||
|
_smolvm.machine_exec(
|
||||||
|
target,
|
||||||
|
[
|
||||||
|
"sh", "-c",
|
||||||
|
f"chown -R {owner_q} {guest_path_q} && "
|
||||||
|
f"chmod {mode_q} {guest_path_q}",
|
||||||
|
],
|
||||||
|
)
|
||||||
@@ -27,11 +27,13 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
|
import time
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Mapping, Sequence
|
from typing import Mapping, Sequence
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
_SMOLVM = "smolvm"
|
_SMOLVM = "smolvm"
|
||||||
|
|
||||||
|
|
||||||
@@ -197,6 +199,34 @@ def machine_exec(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
||||||
|
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
|
||||||
|
|
||||||
|
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
|
||||||
|
channel needs a brief warm-up before back-to-back exec calls are
|
||||||
|
safe. Polling exits as soon as the channel is ready and fails
|
||||||
|
loudly if the VM never responds."""
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
delay = 0.1
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
r = machine_exec(name, ["true"])
|
||||||
|
if r.returncode == 0:
|
||||||
|
return
|
||||||
|
remaining = deadline - time.monotonic()
|
||||||
|
if remaining <= 0:
|
||||||
|
break
|
||||||
|
time.sleep(min(delay, remaining))
|
||||||
|
delay = min(delay * 2, 0.5)
|
||||||
|
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
|
||||||
|
raise SmolvmError(
|
||||||
|
argv,
|
||||||
|
subprocess.CompletedProcess(
|
||||||
|
args=argv, returncode=-1, stdout="",
|
||||||
|
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def machine_cp(src: str, dst: str) -> None:
|
def machine_cp(src: str, dst: str) -> None:
|
||||||
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
||||||
reference a path inside the VM, bare path for the host. Both
|
reference a path inside the VM, bare path for the host. Both
|
||||||
|
|||||||
@@ -7,7 +7,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
from ..log import Die, die
|
from ..log import Die, die, error
|
||||||
|
from ..manifest import ManifestError
|
||||||
from ._common import PROG
|
from ._common import PROG
|
||||||
from . import list as _list_mod
|
from . import list as _list_mod
|
||||||
from .cleanup import cmd_cleanup
|
from .cleanup import cmd_cleanup
|
||||||
@@ -63,6 +64,11 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
die(f"unknown command: {command}")
|
die(f"unknown command: {command}")
|
||||||
try:
|
try:
|
||||||
return handler(rest) or 0
|
return handler(rest) or 0
|
||||||
|
except ManifestError as e:
|
||||||
|
# Manifest/config problems surface as a catchable exception;
|
||||||
|
# print the reason and exit non-zero (same UX die() used to give).
|
||||||
|
error(str(e))
|
||||||
|
return 1
|
||||||
except Die as e:
|
except Die as e:
|
||||||
return e.code if isinstance(e.code, int) else 1
|
return e.code if isinstance(e.code, int) else 1
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
|
|||||||
+89
-18
@@ -21,6 +21,7 @@ import subprocess
|
|||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
import time
|
import time
|
||||||
|
import traceback
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -52,8 +53,8 @@ from ..backend.docker.pipelock_apply import (
|
|||||||
parse_allowlist_content,
|
parse_allowlist_content,
|
||||||
render_allowlist_content,
|
render_allowlist_content,
|
||||||
)
|
)
|
||||||
from ..log import info
|
from ..log import Die, error, info
|
||||||
from ..manifest import Manifest
|
from ..manifest import Manifest, ManifestError
|
||||||
from ..supervise import (
|
from ..supervise import (
|
||||||
ACTION_OPERATOR_EDIT,
|
ACTION_OPERATOR_EDIT,
|
||||||
COMPONENT_FOR_TOOL,
|
COMPONENT_FOR_TOOL,
|
||||||
@@ -174,6 +175,13 @@ def approve(
|
|||||||
qp.proposal.bottle_slug, file_to_apply,
|
qp.proposal.bottle_slug, file_to_apply,
|
||||||
)
|
)
|
||||||
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||||
|
_meta = read_metadata(qp.proposal.bottle_slug)
|
||||||
|
if _meta is not None and not _meta.compose_project:
|
||||||
|
raise CapabilityApplyError(
|
||||||
|
"capability-block remediation is not supported for smolmachines "
|
||||||
|
"bottles. Reject this proposal or handle the capability change "
|
||||||
|
"manually, then restart the bottle."
|
||||||
|
)
|
||||||
diff_before, diff_after = apply_capability_change(
|
diff_before, diff_after = apply_capability_change(
|
||||||
qp.proposal.bottle_slug, file_to_apply,
|
qp.proposal.bottle_slug, file_to_apply,
|
||||||
)
|
)
|
||||||
@@ -639,23 +647,19 @@ def _bottle_for_slug(
|
|||||||
) -> tuple["object", str]:
|
) -> tuple["object", str]:
|
||||||
"""Return `(bottle_handle, prompt_path_hint)` for a re-attach.
|
"""Return `(bottle_handle, prompt_path_hint)` for a re-attach.
|
||||||
If the slug is in `bottles` (dashboard-owned), return the stored
|
If the slug is in `bottles` (dashboard-owned), return the stored
|
||||||
handle directly. Otherwise synthesize a `DockerBottle` from the
|
handle directly. Otherwise synthesize a bottle from the persisted
|
||||||
container name `bot-bottle-<slug>`. For synthesized bottles
|
metadata. The backend field in metadata (PRD 0040) selects Docker
|
||||||
the prompt-file path comes from the manifest's agent if we can
|
or smolmachines; unknown or missing metadata defaults to Docker.
|
||||||
resolve it via metadata.json + the loaded manifest; otherwise
|
|
||||||
the re-attach runs without `--append-system-prompt-file`.
|
|
||||||
|
|
||||||
Returns the empty string for prompt_path_hint when we omit the
|
Returns the empty string for prompt_path_hint when we omit the
|
||||||
flag — the caller passes None to DockerBottle in that case."""
|
flag — the caller passes None to DockerBottle in that case."""
|
||||||
from ..backend.docker.bottle import DockerBottle
|
from ..backend.docker.bottle import DockerBottle
|
||||||
from ..backend.docker.bottle_state import read_metadata
|
from ..backend.docker.bottle_state import read_metadata
|
||||||
|
from ..backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
if slug in bottles:
|
if slug in bottles:
|
||||||
_cm, bottle, _identity = bottles[slug]
|
_cm, bottle, _identity = bottles[slug]
|
||||||
return bottle, ""
|
return bottle, ""
|
||||||
# The container hosting the agent's agent process is named
|
instance_name = f"bot-bottle-{slug}"
|
||||||
# `bot-bottle-<slug>` — set by the compose renderer
|
|
||||||
# (no service suffix on the agent service, by design).
|
|
||||||
container_name = f"bot-bottle-{slug}"
|
|
||||||
prompt_path: str | None = None
|
prompt_path: str | None = None
|
||||||
metadata = read_metadata(slug)
|
metadata = read_metadata(slug)
|
||||||
if metadata is not None and manifest is not None:
|
if metadata is not None and manifest is not None:
|
||||||
@@ -665,11 +669,18 @@ def _bottle_for_slug(
|
|||||||
"BOT_BOTTLE_CONTAINER_HOME", "/home/node",
|
"BOT_BOTTLE_CONTAINER_HOME", "/home/node",
|
||||||
)
|
)
|
||||||
prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
||||||
synth = DockerBottle(
|
backend = metadata.backend if metadata is not None else ""
|
||||||
container=container_name,
|
if backend == "smolmachines":
|
||||||
teardown=lambda: None,
|
synth: object = SmolmachinesBottle(
|
||||||
prompt_path_in_container=prompt_path,
|
instance_name,
|
||||||
)
|
prompt_path=prompt_path,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
synth = DockerBottle(
|
||||||
|
container=instance_name,
|
||||||
|
teardown=lambda: None,
|
||||||
|
prompt_path_in_container=prompt_path,
|
||||||
|
)
|
||||||
return synth, (prompt_path or "")
|
return synth, (prompt_path or "")
|
||||||
|
|
||||||
|
|
||||||
@@ -1277,9 +1288,57 @@ def cmd_dashboard(argv: list[str]) -> int:
|
|||||||
curses.wrapper(_main_loop)
|
curses.wrapper(_main_loop)
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
return 130
|
return 130
|
||||||
|
except Die as e:
|
||||||
|
# die() printed the reason to stderr, but that happened while
|
||||||
|
# curses owned the terminal — the text landed on the alternate
|
||||||
|
# screen and was wiped when the terminal was restored. Re-surface
|
||||||
|
# it now that we're back on the normal screen.
|
||||||
|
if e.message:
|
||||||
|
error(e.message)
|
||||||
|
else:
|
||||||
|
error("dashboard exited on a fatal error (no detail captured).")
|
||||||
|
return e.code if isinstance(e.code, int) else 1
|
||||||
|
except Exception as e:
|
||||||
|
# Any other crash inside the TUI. The traceback would otherwise
|
||||||
|
# vanish with the alternate screen, so persist it and tell the
|
||||||
|
# operator where to look.
|
||||||
|
log_path = _write_crash_log(e)
|
||||||
|
error(f"dashboard crashed: {type(e).__name__}: {e}")
|
||||||
|
error(f"full traceback written to {log_path}")
|
||||||
|
return 1
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _write_crash_log(exc: BaseException) -> Path:
|
||||||
|
"""Persist `exc`'s traceback to a stable file under ~/.bot-bottle/
|
||||||
|
and return its path.
|
||||||
|
|
||||||
|
The dashboard runs under curses, so a crash's stderr/traceback is
|
||||||
|
painted onto the alternate screen and lost when the terminal is
|
||||||
|
restored — this leaves the operator a durable record of *why* it
|
||||||
|
died. Best-effort: falls back to a tempfile if the home dir can't
|
||||||
|
be written."""
|
||||||
|
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
body = "".join(
|
||||||
|
traceback.format_exception(type(exc), exc, exc.__traceback__)
|
||||||
|
)
|
||||||
|
entry = f"=== dashboard crash {stamp} ===\n{body}\n"
|
||||||
|
try:
|
||||||
|
log_dir = _supervise.bot_bottle_root() / "logs"
|
||||||
|
log_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
path = log_dir / "dashboard-crash.log"
|
||||||
|
with path.open("a", encoding="utf-8") as fh:
|
||||||
|
fh.write(entry)
|
||||||
|
return path
|
||||||
|
except OSError:
|
||||||
|
fd, tmp = tempfile.mkstemp(
|
||||||
|
prefix="bot-bottle-dashboard-crash-", suffix=".log",
|
||||||
|
)
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(entry)
|
||||||
|
return Path(tmp)
|
||||||
|
|
||||||
|
|
||||||
def _list_once() -> int:
|
def _list_once() -> int:
|
||||||
pending = discover_pending()
|
pending = discover_pending()
|
||||||
if not pending:
|
if not pending:
|
||||||
@@ -1407,8 +1466,17 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None:
|
|||||||
if manifest_cache[0] is None:
|
if manifest_cache[0] is None:
|
||||||
manifest_cache[0] = Manifest.resolve(USER_CWD, missing_ok=True)
|
manifest_cache[0] = Manifest.resolve(USER_CWD, missing_ok=True)
|
||||||
return manifest_cache[0]
|
return manifest_cache[0]
|
||||||
if not _get_manifest().bottles and not _get_manifest().agents:
|
# A malformed manifest must not take the whole dashboard down — the
|
||||||
status_line = "warning: no bot-bottle config/agents found; new-agent picker is empty"
|
# operator may just be watching running bottles. Degrade to a
|
||||||
|
# status-line warning instead. (Any non-config error propagates to
|
||||||
|
# cmd_dashboard's crash handler.)
|
||||||
|
try:
|
||||||
|
_loaded = _get_manifest()
|
||||||
|
except ManifestError as e:
|
||||||
|
status_line = f"config error: {e}"
|
||||||
|
else:
|
||||||
|
if not _loaded.bottles and not _loaded.agents:
|
||||||
|
status_line = "warning: no bot-bottle config/agents found; new-agent picker is empty"
|
||||||
# First-tick guard: a brand-new dashboard finds any
|
# First-tick guard: a brand-new dashboard finds any
|
||||||
# pre-existing queue entries on its first poll; those
|
# pre-existing queue entries on its first poll; those
|
||||||
# shouldn't ring the bell as if they just arrived.
|
# shouldn't ring the bell as if they just arrived.
|
||||||
@@ -1494,6 +1562,9 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None:
|
|||||||
# bottle running.
|
# bottle running.
|
||||||
try:
|
try:
|
||||||
manifest = _get_manifest()
|
manifest = _get_manifest()
|
||||||
|
except ManifestError as e:
|
||||||
|
status_line = f"config error: {e}"
|
||||||
|
continue
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
status_line = f"manifest load failed: {e}"
|
status_line = f"manifest load failed: {e}"
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -52,8 +52,10 @@ def cmd_resume(argv: list[str]) -> int:
|
|||||||
user_cwd=metadata.cwd or USER_CWD,
|
user_cwd=metadata.cwd or USER_CWD,
|
||||||
identity=metadata.identity,
|
identity=metadata.identity,
|
||||||
)
|
)
|
||||||
|
backend_name = metadata.backend or None
|
||||||
return _launch_bottle(
|
return _launch_bottle(
|
||||||
spec,
|
spec,
|
||||||
dry_run=args.dry_run,
|
dry_run=args.dry_run,
|
||||||
remote_control=args.remote_control,
|
remote_control=args.remote_control,
|
||||||
|
backend_name=backend_name,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,325 @@
|
|||||||
|
"""Host Codex auth helpers.
|
||||||
|
|
||||||
|
Reads the host's Codex ChatGPT/device-login auth state and returns only
|
||||||
|
the short-lived access token needed by egress. This module deliberately
|
||||||
|
does not expose refresh tokens or raw auth payloads.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from copy import deepcopy
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .log import die
|
||||||
|
from .util import expand_tilde
|
||||||
|
|
||||||
|
|
||||||
|
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
|
||||||
|
env = os.environ if host_env is None else host_env
|
||||||
|
home = env.get("CODEX_HOME")
|
||||||
|
if home:
|
||||||
|
return Path(expand_tilde(home)) / "auth.json"
|
||||||
|
return Path.home() / ".codex" / "auth.json"
|
||||||
|
|
||||||
|
|
||||||
|
def codex_host_access_token(
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> str:
|
||||||
|
path = codex_auth_path(host_env)
|
||||||
|
if not path.is_file():
|
||||||
|
die(
|
||||||
|
f"codex host credentials: auth file missing at {path}. "
|
||||||
|
"Run `codex login --device-auth` on the host or disable "
|
||||||
|
"agent_provider.forward_host_credentials."
|
||||||
|
)
|
||||||
|
raw = _read_auth_object(path)
|
||||||
|
|
||||||
|
auth_mode = raw.get("auth_mode")
|
||||||
|
if not isinstance(auth_mode, str) or auth_mode == "api_key":
|
||||||
|
die(
|
||||||
|
"codex host credentials: host Codex auth is not user/device "
|
||||||
|
"auth. Run `codex login --device-auth` on the host."
|
||||||
|
)
|
||||||
|
|
||||||
|
tokens = raw.get("tokens")
|
||||||
|
if not isinstance(tokens, dict):
|
||||||
|
die(f"codex host credentials: {path} is missing tokens")
|
||||||
|
access = tokens.get("access_token")
|
||||||
|
if not isinstance(access, str) or not access:
|
||||||
|
die(
|
||||||
|
f"codex host credentials: {path} is missing tokens.access_token. "
|
||||||
|
"Run `codex login --device-auth` on the host."
|
||||||
|
)
|
||||||
|
|
||||||
|
exp = _jwt_exp(access)
|
||||||
|
if exp is None:
|
||||||
|
die("codex host credentials: tokens.access_token is not a JWT with exp")
|
||||||
|
check_now = now or datetime.now(timezone.utc)
|
||||||
|
if exp <= check_now:
|
||||||
|
die(
|
||||||
|
"codex host credentials: host Codex access token is expired. "
|
||||||
|
"Run `codex login --device-auth` on the host and restart the bottle."
|
||||||
|
)
|
||||||
|
return access
|
||||||
|
|
||||||
|
|
||||||
|
def codex_dummy_auth_json(
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Return a non-secret `auth.json` that keeps Codex in the host's
|
||||||
|
auth branch while egress owns the real bearer token.
|
||||||
|
|
||||||
|
The dummy access/id tokens carry the *host* token's real `exp` so
|
||||||
|
Codex's proactive refresh lifecycle (it refreshes when its local
|
||||||
|
access token is at/past expiry) tracks the real token instead of
|
||||||
|
firing after an artificial TTL. Codex cannot refresh inside the
|
||||||
|
bottle — the refresh token is a placeholder and the OpenAI token
|
||||||
|
endpoint is off-route — so a shorter dummy exp would drop Codex to
|
||||||
|
the sign-in screen the moment it lapsed, even while egress still
|
||||||
|
holds a valid bearer."""
|
||||||
|
path = codex_auth_path(host_env)
|
||||||
|
access = codex_host_access_token(host_env, now=now)
|
||||||
|
raw = _read_auth_object(path)
|
||||||
|
host_exp = _jwt_exp(access)
|
||||||
|
exp_ts = int(host_exp.timestamp()) if host_exp is not None else None
|
||||||
|
dummy = _redact_codex_auth(deepcopy(raw), now=now, exp_ts=exp_ts)
|
||||||
|
return json.dumps(dummy, indent=2, sort_keys=True) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def write_codex_dummy_auth_file(
|
||||||
|
path: Path,
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(codex_dummy_auth_json(host_env, now=now))
|
||||||
|
path.chmod(0o600)
|
||||||
|
|
||||||
|
|
||||||
|
def _read_auth_object(path: Path) -> dict:
|
||||||
|
try:
|
||||||
|
raw = json.loads(path.read_text())
|
||||||
|
except (OSError, json.JSONDecodeError) as e:
|
||||||
|
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
die(f"codex host credentials: {path} must contain a JSON object")
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
||||||
|
if exp_ts is not None:
|
||||||
|
return exp_ts
|
||||||
|
check_now = now or datetime.now(timezone.utc)
|
||||||
|
return int(check_now.timestamp()) + 3600
|
||||||
|
|
||||||
|
|
||||||
|
def _dummy_timestamp(now: datetime | None = None) -> str:
|
||||||
|
check_now = now or datetime.now(timezone.utc)
|
||||||
|
if check_now.tzinfo is None:
|
||||||
|
check_now = check_now.replace(tzinfo=timezone.utc)
|
||||||
|
check_now = check_now.astimezone(timezone.utc)
|
||||||
|
return check_now.isoformat(timespec="milliseconds").replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
|
||||||
|
return _encode_dummy_jwt({
|
||||||
|
"exp": _dummy_exp(now, exp_ts),
|
||||||
|
"sub": "bot-bottle-placeholder",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def _dummy_jwt_from_host(
|
||||||
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
|
) -> str:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
|
parts = value.split(".")
|
||||||
|
if len(parts) < 2:
|
||||||
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
|
try:
|
||||||
|
payload = json.loads(_b64url_decode(parts[1]))
|
||||||
|
except (ValueError, json.JSONDecodeError):
|
||||||
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
|
return _encode_dummy_jwt(_redact_jwt_payload(payload, now=now, exp_ts=exp_ts))
|
||||||
|
|
||||||
|
|
||||||
|
def _encode_dummy_jwt(payload: dict) -> str:
|
||||||
|
def enc(obj: dict) -> str:
|
||||||
|
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||||
|
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||||
|
|
||||||
|
return f"{enc({'alg': 'none', 'typ': 'JWT'})}.{enc(payload)}.placeholder"
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_jwt_payload(
|
||||||
|
payload: dict,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
exp_ts: int | None = None,
|
||||||
|
) -> dict:
|
||||||
|
out = _redact_claims(payload)
|
||||||
|
if not isinstance(out, dict):
|
||||||
|
out = {}
|
||||||
|
out["exp"] = _dummy_exp(now, exp_ts)
|
||||||
|
out.setdefault("sub", "bot-bottle-placeholder")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_claims(value: object) -> object:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
out: dict[str, object] = {}
|
||||||
|
for key, inner in value.items():
|
||||||
|
lower = key.lower()
|
||||||
|
if key == "https://api.openai.com/profile":
|
||||||
|
out[key] = _redact_profile_claim(inner)
|
||||||
|
elif key == "https://api.openai.com/auth":
|
||||||
|
out[key] = _redact_auth_claim(inner)
|
||||||
|
elif lower == "email":
|
||||||
|
out[key] = "bot-bottle@example.invalid"
|
||||||
|
elif lower == "email_verified":
|
||||||
|
out[key] = True
|
||||||
|
elif lower in {"exp", "iat", "nbf", "auth_time", "pwd_auth_time"}:
|
||||||
|
out[key] = inner if isinstance(inner, (int, float)) else 0
|
||||||
|
elif lower in {"aud", "scp", "amr"}:
|
||||||
|
out[key] = inner if isinstance(inner, list) else []
|
||||||
|
elif isinstance(inner, bool):
|
||||||
|
out[key] = inner
|
||||||
|
elif isinstance(inner, dict):
|
||||||
|
out[key] = {}
|
||||||
|
elif isinstance(inner, list):
|
||||||
|
out[key] = []
|
||||||
|
else:
|
||||||
|
out[key] = "bot-bottle-placeholder"
|
||||||
|
return out
|
||||||
|
if isinstance(value, list):
|
||||||
|
return []
|
||||||
|
return "bot-bottle-placeholder"
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_profile_claim(value: object) -> dict:
|
||||||
|
profile = value if isinstance(value, dict) else {}
|
||||||
|
return {
|
||||||
|
"email": "bot-bottle@example.invalid",
|
||||||
|
"email_verified": bool(profile.get("email_verified", True)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_auth_claim(value: object) -> dict:
|
||||||
|
auth = value if isinstance(value, dict) else {}
|
||||||
|
out: dict[str, object] = {}
|
||||||
|
for key, inner in auth.items():
|
||||||
|
lower = key.lower()
|
||||||
|
if lower == "chatgpt_plan_type" and isinstance(inner, str) and inner:
|
||||||
|
out[key] = inner
|
||||||
|
elif lower == "chatgpt_account_id" and isinstance(inner, str) and inner:
|
||||||
|
# Current Codex uses the selected account id when building
|
||||||
|
# ChatGPT requests. Keep that non-secret identifier aligned
|
||||||
|
# with the host while egress owns the real bearer token.
|
||||||
|
out[key] = inner
|
||||||
|
elif lower == "localhost" and isinstance(inner, bool):
|
||||||
|
out[key] = inner
|
||||||
|
elif isinstance(inner, bool):
|
||||||
|
out[key] = inner
|
||||||
|
elif isinstance(inner, list):
|
||||||
|
out[key] = []
|
||||||
|
elif isinstance(inner, dict):
|
||||||
|
out[key] = {}
|
||||||
|
else:
|
||||||
|
out[key] = "bot-bottle-placeholder"
|
||||||
|
out.setdefault("chatgpt_plan_type", "unknown")
|
||||||
|
out.setdefault("user_id", "bot-bottle-placeholder")
|
||||||
|
out.setdefault("chatgpt_user_id", "bot-bottle-placeholder")
|
||||||
|
out.setdefault("chatgpt_account_id", "bot-bottle-placeholder")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_codex_auth(
|
||||||
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
|
) -> object:
|
||||||
|
auth = value if isinstance(value, dict) else {}
|
||||||
|
out: dict[str, object] = {}
|
||||||
|
for key, inner in auth.items():
|
||||||
|
lower = key.lower()
|
||||||
|
if lower == "auth_mode" and isinstance(inner, str) and inner:
|
||||||
|
out[key] = inner
|
||||||
|
elif lower == "openai_api_key":
|
||||||
|
out[key] = None
|
||||||
|
elif lower == "last_refresh":
|
||||||
|
# Codex parses this as a timestamp on startup. Keep the
|
||||||
|
# schema valid without copying host-side session metadata.
|
||||||
|
out[key] = _dummy_timestamp(now)
|
||||||
|
elif lower == "tokens":
|
||||||
|
out[key] = _redact_token_block(inner, now=now, exp_ts=exp_ts)
|
||||||
|
else:
|
||||||
|
out[key] = _redact_unknown_auth_value(inner)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_token_block(
|
||||||
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
tokens = value if isinstance(value, dict) else {}
|
||||||
|
out: dict[str, object] = {}
|
||||||
|
for key, inner in tokens.items():
|
||||||
|
lower = key.lower()
|
||||||
|
if lower in {"access_token", "id_token"}:
|
||||||
|
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
|
||||||
|
elif lower == "account_id" and isinstance(inner, str) and inner:
|
||||||
|
# Current Codex uses this non-secret selected account id
|
||||||
|
# while egress owns the real bearer token.
|
||||||
|
out[key] = inner
|
||||||
|
else:
|
||||||
|
out[key] = _redact_unknown_auth_value(inner)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_unknown_auth_value(value: object) -> object:
|
||||||
|
if isinstance(value, bool):
|
||||||
|
return value
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return []
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
return "bot-bottle-placeholder"
|
||||||
|
|
||||||
|
|
||||||
|
def _jwt_exp(token: str) -> datetime | None:
|
||||||
|
parts = token.split(".")
|
||||||
|
if len(parts) < 2:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
payload = json.loads(_b64url_decode(parts[1]))
|
||||||
|
except (ValueError, json.JSONDecodeError):
|
||||||
|
return None
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return None
|
||||||
|
exp = payload.get("exp")
|
||||||
|
if not isinstance(exp, (int, float)):
|
||||||
|
return None
|
||||||
|
return datetime.fromtimestamp(exp, timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _b64url_decode(value: str) -> str:
|
||||||
|
padded = value + ("=" * (-len(value) % 4))
|
||||||
|
return base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"codex_auth_path",
|
||||||
|
"codex_dummy_auth_json",
|
||||||
|
"codex_host_access_token",
|
||||||
|
"write_codex_dummy_auth_file",
|
||||||
|
]
|
||||||
+110
-86
@@ -24,12 +24,19 @@ flow (PRD 0014) at egress and renames the MCP tool.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from .egress_addon_core import Route
|
||||||
from .log import die
|
from .log import die
|
||||||
from .manifest import Bottle
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from .manifest import Bottle
|
||||||
|
|
||||||
|
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||||
|
|
||||||
|
|
||||||
# DNS name agents will dial for the per-bottle egress sidecar.
|
# DNS name agents will dial for the per-bottle egress sidecar.
|
||||||
@@ -48,32 +55,30 @@ EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
|||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class EgressRoute:
|
class EgressRoute(Route):
|
||||||
"""One resolved route on the egress sidecar.
|
"""Host-side extension of the addon's `Route`.
|
||||||
|
|
||||||
`host` matches the request's hostname (case-insensitive). The
|
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
|
||||||
optional `path_allowlist` constrains the URL path; empty tuple
|
from `egress_addon_core.Route` — those are the fields that cross the
|
||||||
means no path-level filtering. The `auth_scheme` / `token_env` /
|
YAML wire into the sidecar. The three fields below are host-only and
|
||||||
`token_ref` triple is the credential-injection config; empty
|
are never serialised to the addon.
|
||||||
strings mean "no auth injection" (the manifest's nested `auth`
|
|
||||||
block was omitted).
|
|
||||||
|
|
||||||
`token_env` is the env-var slot inside the egress container
|
`token_ref` is the host env var the CLI reads at launch and forwards
|
||||||
(e.g. `EGRESS_TOKEN_0`); `token_ref` is the host env var
|
into the container's environ under `token_env`. Routes that share a
|
||||||
the CLI reads at launch and forwards into the container's environ
|
`token_ref` coalesce to one `token_env` slot.
|
||||||
under `token_env`. Routes that share a `token_ref` coalesce to
|
|
||||||
one `token_env` slot.
|
|
||||||
|
|
||||||
`roles` carries the manifest route's optional role markers (see
|
`roles` carries the manifest route's role tuple (reserved for
|
||||||
`manifest.EGRESS_ROLES`). The launch step reads these for
|
future use; always empty today).
|
||||||
side effects like the claude-code OAuth placeholder env."""
|
|
||||||
|
`tls_passthrough` signals that pipelock must not TLS-MITM this
|
||||||
|
host — either because the manifest declared `pipelock.tls_passthrough:
|
||||||
|
true` (lifted in `egress_manifest_routes`) or because a provider
|
||||||
|
route set it (e.g. egress injects its own Bearer on that host
|
||||||
|
after the agent boundary and pipelock's header DLP would block it)."""
|
||||||
|
|
||||||
host: str
|
|
||||||
path_allowlist: tuple[str, ...] = ()
|
|
||||||
auth_scheme: str = ""
|
|
||||||
token_env: str = ""
|
|
||||||
token_ref: str = ""
|
token_ref: str = ""
|
||||||
roles: tuple[str, ...] = ()
|
roles: tuple[str, ...] = ()
|
||||||
|
tls_passthrough: bool = False
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -130,55 +135,60 @@ class EgressPlan:
|
|||||||
def egress_manifest_routes(
|
def egress_manifest_routes(
|
||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Lift each `bottle.egress.routes[]` manifest entry into a
|
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
|
||||||
resolved EgressRoute. Order is preserved so route lookup at
|
Order is preserved. Token slots are not assigned here — slot assignment
|
||||||
the proxy is stable.
|
is a final step in `egress_routes_for_bottle` after provider and manifest
|
||||||
|
routes are merged."""
|
||||||
Token-env slots are assigned per distinct `token_ref`: the first
|
|
||||||
authenticated route with `token_ref` "GH_PAT" gets
|
|
||||||
`EGRESS_TOKEN_0`; a second route with the same `token_ref`
|
|
||||||
shares slot 0. Unauthenticated routes (`auth` omitted) contribute
|
|
||||||
no slot.
|
|
||||||
|
|
||||||
This is the effective set the addon enforces. Provider runtime
|
|
||||||
routes are intentionally not injected implicitly; every allowed
|
|
||||||
host must come from the home-owned bottle manifest."""
|
|
||||||
out: list[EgressRoute] = []
|
out: list[EgressRoute] = []
|
||||||
slot_for_token: dict[str, str] = {}
|
|
||||||
for r in bottle.egress.routes:
|
for r in bottle.egress.routes:
|
||||||
if r.AuthScheme and r.TokenRef:
|
out.append(EgressRoute(
|
||||||
token_env = slot_for_token.get(r.TokenRef)
|
host=r.Host,
|
||||||
if token_env is None:
|
path_allowlist=r.PathAllowlist,
|
||||||
token_env = f"EGRESS_TOKEN_{len(slot_for_token)}"
|
auth_scheme=r.AuthScheme,
|
||||||
slot_for_token[r.TokenRef] = token_env
|
token_ref=r.TokenRef,
|
||||||
out.append(EgressRoute(
|
roles=r.Role,
|
||||||
host=r.Host,
|
tls_passthrough=r.Pipelock.TlsPassthrough,
|
||||||
path_allowlist=r.PathAllowlist,
|
))
|
||||||
auth_scheme=r.AuthScheme,
|
|
||||||
token_env=token_env,
|
|
||||||
token_ref=r.TokenRef,
|
|
||||||
roles=r.Role,
|
|
||||||
))
|
|
||||||
else:
|
|
||||||
out.append(EgressRoute(
|
|
||||||
host=r.Host,
|
|
||||||
path_allowlist=r.PathAllowlist,
|
|
||||||
roles=r.Role,
|
|
||||||
))
|
|
||||||
return tuple(out)
|
return tuple(out)
|
||||||
|
|
||||||
|
|
||||||
def egress_routes_for_bottle(
|
def egress_routes_for_bottle(
|
||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Effective egress routes. This is what gets rendered into
|
"""Effective egress routes for the agent.
|
||||||
routes.yaml + what the addon enforces.
|
|
||||||
|
|
||||||
Operators that want to allow a host declare it directly in
|
Provider routes own their hosts outright; manifest routes for hosts
|
||||||
`bottle.egress.routes` as an authenticated route or bare-pass entry
|
not claimed by any provider are appended. Token slots are assigned
|
||||||
(`- host: <name>`). The legacy `bottle.egress.allowlist`
|
in a final pass over the merged list in order, so provisioned routes
|
||||||
folding is gone — egress is the single allowlist surface."""
|
get the lower slot numbers."""
|
||||||
return egress_manifest_routes(bottle)
|
manifest = egress_manifest_routes(bottle)
|
||||||
|
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||||
|
merged = list(provider_routes) + [
|
||||||
|
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||||
|
]
|
||||||
|
return _assign_token_slots(merged)
|
||||||
|
|
||||||
|
|
||||||
|
def _assign_token_slots(
|
||||||
|
routes: list[EgressRoute],
|
||||||
|
) -> tuple[EgressRoute, ...]:
|
||||||
|
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
|
||||||
|
|
||||||
|
Routes sharing a token_ref share a slot. Unauthenticated routes
|
||||||
|
(no auth_scheme / token_ref) keep token_env empty."""
|
||||||
|
slot_for_ref: dict[str, str] = {}
|
||||||
|
out: list[EgressRoute] = []
|
||||||
|
for r in routes:
|
||||||
|
if r.auth_scheme and r.token_ref:
|
||||||
|
slot = slot_for_ref.get(r.token_ref)
|
||||||
|
if slot is None:
|
||||||
|
slot = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
||||||
|
slot_for_ref[r.token_ref] = slot
|
||||||
|
out.append(dataclasses.replace(r, token_env=slot))
|
||||||
|
else:
|
||||||
|
out.append(r)
|
||||||
|
return tuple(out)
|
||||||
|
|
||||||
|
|
||||||
def egress_token_env_map(
|
def egress_token_env_map(
|
||||||
@@ -193,7 +203,7 @@ def egress_token_env_map(
|
|||||||
silently picking one."""
|
silently picking one."""
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
for r in routes:
|
for r in routes:
|
||||||
if not r.token_env:
|
if not (r.auth_scheme and r.token_ref and r.token_env):
|
||||||
continue
|
continue
|
||||||
existing = out.get(r.token_env)
|
existing = out.get(r.token_env)
|
||||||
if existing is not None and existing != r.token_ref:
|
if existing is not None and existing != r.token_ref:
|
||||||
@@ -206,35 +216,43 @@ def egress_token_env_map(
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _route_to_yaml_fields(r: Route) -> dict:
|
||||||
|
"""Return the addon-visible fields for one route.
|
||||||
|
|
||||||
|
Single authoritative mapping between EgressRoute (host-side) and
|
||||||
|
egress_addon_core.Route (sidecar-side). When a field is added to
|
||||||
|
the addon's Route that must appear in the YAML, add it here and
|
||||||
|
in egress_addon_core._parse_one together."""
|
||||||
|
fields: dict = {"host": r.host}
|
||||||
|
if r.auth_scheme and r.token_env:
|
||||||
|
fields["auth_scheme"] = r.auth_scheme
|
||||||
|
fields["token_env"] = r.token_env
|
||||||
|
if r.path_allowlist:
|
||||||
|
fields["path_allowlist"] = list(r.path_allowlist)
|
||||||
|
return fields
|
||||||
|
|
||||||
|
|
||||||
def egress_render_routes(
|
def egress_render_routes(
|
||||||
routes: tuple[EgressRoute, ...],
|
routes: tuple[EgressRoute, ...],
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Serialize the route table for the addon to read.
|
"""Serialize the route table for the addon to read.
|
||||||
|
|
||||||
YAML content — no token values, no host env-var names. The only
|
YAML content — no token values, no host env-var names. Fields are
|
||||||
thing the addon needs at runtime is the host → path_allowlist
|
determined by `_route_to_yaml_fields`, which is the single point of
|
||||||
+ auth_scheme + in-container env-var mapping. The actual token
|
truth for the EgressRoute → egress_addon_core.Route mapping."""
|
||||||
values arrive via the container's environ.
|
|
||||||
|
|
||||||
Authenticated routes carry `auth_scheme` + `token_env`;
|
|
||||||
unauthenticated routes omit both keys (the addon's parser
|
|
||||||
enforces both-or-neither). Hand-rolled YAML in the style of
|
|
||||||
`pipelock_render_yaml` so the addon's parser
|
|
||||||
(`yaml_subset.parse_yaml_subset`) round-trips it cleanly."""
|
|
||||||
lines: list[str] = ["routes:"]
|
lines: list[str] = ["routes:"]
|
||||||
if not routes:
|
if not routes:
|
||||||
# `routes:` with an empty list on the same line — the parser
|
|
||||||
# needs SOMETHING here. Empty inline list is the cleanest.
|
|
||||||
lines[0] = "routes: []"
|
lines[0] = "routes: []"
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
for r in routes:
|
for r in routes:
|
||||||
lines.append(f' - host: "{r.host}"')
|
f = _route_to_yaml_fields(r)
|
||||||
if r.auth_scheme and r.token_env:
|
lines.append(f' - host: "{f["host"]}"')
|
||||||
lines.append(f' auth_scheme: "{r.auth_scheme}"')
|
if "auth_scheme" in f:
|
||||||
lines.append(f' token_env: "{r.token_env}"')
|
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||||
if r.path_allowlist:
|
lines.append(f' token_env: "{f["token_env"]}"')
|
||||||
|
if "path_allowlist" in f:
|
||||||
lines.append(" path_allowlist:")
|
lines.append(" path_allowlist:")
|
||||||
for p in r.path_allowlist:
|
for p in f["path_allowlist"]:
|
||||||
lines.append(f' - "{p}"')
|
lines.append(f' - "{p}"')
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
@@ -274,18 +292,23 @@ class Egress(ABC):
|
|||||||
sidecar's start/stop lifecycle is backend-specific and lives on
|
sidecar's start/stop lifecycle is backend-specific and lives on
|
||||||
concrete subclasses."""
|
concrete subclasses."""
|
||||||
|
|
||||||
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> EgressPlan:
|
def prepare(
|
||||||
"""Lift `bottle.egress.routes` into resolved routes,
|
self,
|
||||||
render the routes file (mode 600) under `stage_dir`, and
|
bottle: Bottle,
|
||||||
|
slug: str,
|
||||||
|
stage_dir: Path,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
|
) -> EgressPlan:
|
||||||
|
"""Lift `bottle.egress.routes` + `provider_routes` into resolved
|
||||||
|
routes, render the routes file (mode 600) under `stage_dir`, and
|
||||||
return the plan. Pure host-side, no docker subprocess. The
|
return the plan. Pure host-side, no docker subprocess. The
|
||||||
token-env map records the mapping the launch step uses to
|
token-env map records the mapping the launch step uses to
|
||||||
forward values from the host's environ into the sidecar's
|
forward values from the host's environ into the sidecar's environ.
|
||||||
environ.
|
|
||||||
|
|
||||||
Returned plan is incomplete: the launch step must fill
|
Returned plan is incomplete: the launch step must fill
|
||||||
`internal_network` / `egress_network` / `pipelock_proxy_url`
|
`internal_network` / `egress_network` / `pipelock_proxy_url`
|
||||||
via `dataclasses.replace` before passing it to `.start`."""
|
via `dataclasses.replace` before passing it to `.start`."""
|
||||||
routes = egress_routes_for_bottle(bottle)
|
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||||
routes_path = stage_dir / "egress_routes.yaml"
|
routes_path = stage_dir / "egress_routes.yaml"
|
||||||
routes_path.write_text(egress_render_routes(routes))
|
routes_path.write_text(egress_render_routes(routes))
|
||||||
routes_path.chmod(0o600)
|
routes_path.chmod(0o600)
|
||||||
@@ -297,6 +320,7 @@ class Egress(ABC):
|
|||||||
)
|
)
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
|
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
|
||||||
"EGRESS_HOSTNAME",
|
"EGRESS_HOSTNAME",
|
||||||
"EGRESS_ROUTES_IN_CONTAINER",
|
"EGRESS_ROUTES_IN_CONTAINER",
|
||||||
"Egress",
|
"Egress",
|
||||||
|
|||||||
+24
-50
@@ -29,22 +29,21 @@ backend-specific and lives on concrete subclasses (see
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shlex
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Mapping
|
|
||||||
|
|
||||||
from .log import die
|
|
||||||
from .manifest import Bottle, GitEntry
|
from .manifest import Bottle, GitEntry
|
||||||
|
|
||||||
|
|
||||||
# Short network alias for git-gate inside the sidecar bundle. The
|
# Short network alias for git-gate inside the sidecar bundle. The
|
||||||
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
|
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
|
||||||
GIT_GATE_HOSTNAME = "git-gate"
|
GIT_GATE_HOSTNAME = "git-gate"
|
||||||
|
# Bound half-open git client sessions. If an agent/tool runner is
|
||||||
|
# interrupted during push, git daemon should reap the receive-pack
|
||||||
def _empty_str_map() -> dict[str, str]:
|
# child instead of keeping the gate wedged indefinitely.
|
||||||
return {}
|
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -60,10 +59,7 @@ class GitGateUpstream:
|
|||||||
KnownHostKey string from the manifest; the gate's start step
|
KnownHostKey string from the manifest; the gate's start step
|
||||||
materialises it into a known_hosts file if non-empty.
|
materialises it into a known_hosts file if non-empty.
|
||||||
|
|
||||||
`extra_hosts` is a `{hostname: ip}` map the backend injects into
|
the gate credential paths inside the running sidecar."""
|
||||||
the gate container's `/etc/hosts` via `--add-host` so the gate
|
|
||||||
can resolve upstream hostnames that aren't reachable via the
|
|
||||||
container's default DNS (e.g. Tailscale-only hosts)."""
|
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
upstream_url: str
|
upstream_url: str
|
||||||
@@ -72,7 +68,6 @@ class GitGateUpstream:
|
|||||||
identity_file: str
|
identity_file: str
|
||||||
known_host_key: str
|
known_host_key: str
|
||||||
known_hosts_file: Path = Path()
|
known_hosts_file: Path = Path()
|
||||||
extra_hosts: Mapping[str, str] = field(default_factory=_empty_str_map)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -109,46 +104,19 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
|
|||||||
upstream_port=e.UpstreamPort,
|
upstream_port=e.UpstreamPort,
|
||||||
identity_file=e.IdentityFile,
|
identity_file=e.IdentityFile,
|
||||||
known_host_key=e.KnownHostKey,
|
known_host_key=e.KnownHostKey,
|
||||||
extra_hosts=dict(e.ExtraHosts),
|
|
||||||
)
|
)
|
||||||
for e in bottle.git
|
for e in bottle.git
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def git_gate_aggregate_extra_hosts(
|
|
||||||
upstreams: tuple[GitGateUpstream, ...],
|
|
||||||
) -> dict[str, str]:
|
|
||||||
"""Merge every upstream's `extra_hosts` into a single
|
|
||||||
`{hostname: ip}` map for `--add-host` on the gate container. Two
|
|
||||||
entries naming the same hostname with different IPs is a manifest
|
|
||||||
bug — the gate has one /etc/hosts — so die loudly with the
|
|
||||||
conflicting names rather than silently picking one."""
|
|
||||||
merged: dict[str, str] = {}
|
|
||||||
source: dict[str, str] = {}
|
|
||||||
for u in upstreams:
|
|
||||||
for host, ip in u.extra_hosts.items():
|
|
||||||
existing = merged.get(host)
|
|
||||||
if existing is None:
|
|
||||||
merged[host] = ip
|
|
||||||
source[host] = u.name
|
|
||||||
elif existing != ip:
|
|
||||||
die(
|
|
||||||
f"git-gate ExtraHosts conflict: '{host}' maps to "
|
|
||||||
f"'{existing}' in upstream '{source[host]}' and to "
|
|
||||||
f"'{ip}' in upstream '{u.name}'. The gate has one "
|
|
||||||
f"/etc/hosts; pick one IP."
|
|
||||||
)
|
|
||||||
return merged
|
|
||||||
|
|
||||||
|
|
||||||
def git_gate_render_gitconfig(
|
def git_gate_render_gitconfig(
|
||||||
entries: tuple[GitEntry, ...], gate_host: str
|
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Render the agent's ~/.gitconfig content for git-gate
|
"""Render the agent's ~/.gitconfig content for git-gate
|
||||||
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
|
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
|
||||||
exposed for tests + reuse across backends.
|
exposed for tests + reuse across backends.
|
||||||
|
|
||||||
`gate_host` is the part of the URL between `git://` and the
|
`gate_host` is the part of the URL between `<scheme>://` and the
|
||||||
repo path — backends differ here:
|
repo path — backends differ here:
|
||||||
- docker: `git-gate` (the short network alias)
|
- docker: `git-gate` (the short network alias)
|
||||||
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
|
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
|
||||||
@@ -165,7 +133,7 @@ def git_gate_render_gitconfig(
|
|||||||
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
|
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
|
||||||
]
|
]
|
||||||
for entry in entries:
|
for entry in entries:
|
||||||
out.append(f'[url "git://{gate_host}/{entry.Name}.git"]\n')
|
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
|
||||||
out.append(f"\tinsteadOf = {entry.Upstream}\n")
|
out.append(f"\tinsteadOf = {entry.Upstream}\n")
|
||||||
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
|
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
|
||||||
port = (
|
port = (
|
||||||
@@ -233,20 +201,20 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
|
|||||||
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
|
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
|
||||||
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
|
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
|
||||||
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
|
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
|
||||||
|
" git -C \"$repo\" config http.receivepack true",
|
||||||
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
|
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
|
||||||
"}",
|
"}",
|
||||||
"",
|
"",
|
||||||
"mkdir -p /git",
|
"mkdir -p /git",
|
||||||
]
|
]
|
||||||
for u in upstreams:
|
for u in upstreams:
|
||||||
# Single-quote args so URL/path content (containing : and /)
|
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
|
||||||
# passes through ash unmangled. Names came through the manifest
|
|
||||||
# validator so they don't contain a single quote.
|
|
||||||
lines.append(f"init_repo '{u.name}' '{u.upstream_url}'")
|
|
||||||
lines.extend([
|
lines.extend([
|
||||||
"",
|
"",
|
||||||
"exec git daemon \\",
|
"exec git daemon \\",
|
||||||
" --reuseaddr \\",
|
" --reuseaddr \\",
|
||||||
|
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
|
||||||
|
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
|
||||||
" --base-path=/git \\",
|
" --base-path=/git \\",
|
||||||
" --export-all \\",
|
" --export-all \\",
|
||||||
" --enable=receive-pack \\",
|
" --enable=receive-pack \\",
|
||||||
@@ -280,7 +248,14 @@ while IFS=' ' read -r old new ref; do
|
|||||||
[ -z "$ref" ] && continue
|
[ -z "$ref" ] && continue
|
||||||
[ "$new" = "$zero" ] && continue
|
[ "$new" = "$zero" ] && continue
|
||||||
if [ "$old" = "$zero" ]; then
|
if [ "$old" = "$zero" ]; then
|
||||||
log_opts="$new"
|
# New ref: scan only the commits this push introduces — those
|
||||||
|
# reachable from $new but not from any ref the gate already has.
|
||||||
|
# Everything already on the gate arrived via upstream mirror-fetch
|
||||||
|
# or a previously gitleaks-scanned push, so it's already-upstream
|
||||||
|
# or already-scanned; re-scanning it (the old `$new` full-ancestry
|
||||||
|
# range) only resurfaces historical findings and blocks every new
|
||||||
|
# branch. See PRD 0028 / issue #106.
|
||||||
|
log_opts="$new --not --all"
|
||||||
else
|
else
|
||||||
log_opts="$old..$new"
|
log_opts="$old..$new"
|
||||||
fi
|
fi
|
||||||
@@ -300,7 +275,7 @@ if [ ! -f "$hostsfile" ]; then
|
|||||||
echo "git-gate: add KnownHostKey to the bottle.git entry and restart the bottle" >&2
|
echo "git-gate: add KnownHostKey to the bottle.git entry and restart the bottle" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes"
|
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
|
||||||
|
|
||||||
while IFS=' ' read -r old new ref; do
|
while IFS=' ' read -r old new ref; do
|
||||||
[ -z "$ref" ] && continue
|
[ -z "$ref" ] && continue
|
||||||
@@ -355,7 +330,7 @@ if [ -z "$keyfile" ] || [ ! -f "$hostsfile" ]; then
|
|||||||
echo "git-gate: missing credentials for $repo_dir; refusing fetch" >&2
|
echo "git-gate: missing credentials for $repo_dir; refusing fetch" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes"
|
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
|
||||||
|
|
||||||
echo "git-gate: refreshing $repo_dir from upstream" >&2
|
echo "git-gate: refreshing $repo_dir from upstream" >&2
|
||||||
if ! GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" fetch origin --prune >&2; then
|
if ! GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" fetch origin --prune >&2; then
|
||||||
@@ -429,7 +404,6 @@ class GitGate(ABC):
|
|||||||
identity_file=u.identity_file,
|
identity_file=u.identity_file,
|
||||||
known_host_key=u.known_host_key,
|
known_host_key=u.known_host_key,
|
||||||
known_hosts_file=known_hosts_file,
|
known_hosts_file=known_hosts_file,
|
||||||
extra_hosts=dict(u.extra_hosts),
|
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return GitGatePlan(
|
return GitGatePlan(
|
||||||
|
|||||||
@@ -0,0 +1,175 @@
|
|||||||
|
"""Tiny smart-HTTP wrapper for git-gate repos.
|
||||||
|
|
||||||
|
Used by the smolmachines backend where `git://` push traffic over the
|
||||||
|
host-published Docker port can hang before receive-pack reaches hooks.
|
||||||
|
The wrapper serves the same `/git/*.git` bare repos through
|
||||||
|
`git http-backend`, so pre-receive and upstream forwarding remain the
|
||||||
|
git-gate enforcement point.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_PORT = 9420
|
||||||
|
|
||||||
|
# Body-size cap matching supervise_server.py's 1 MiB limit.
|
||||||
|
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||||
|
|
||||||
|
|
||||||
|
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "bot-bottle-git-http/1"
|
||||||
|
|
||||||
|
def do_GET(self) -> None:
|
||||||
|
self._run_backend()
|
||||||
|
|
||||||
|
def do_POST(self) -> None:
|
||||||
|
self._run_backend()
|
||||||
|
|
||||||
|
def _run_backend(self) -> None:
|
||||||
|
parsed = urlsplit(self.path)
|
||||||
|
if self._is_upload_pack(parsed.path, parsed.query):
|
||||||
|
repo_dir = self._repo_dir(parsed.path)
|
||||||
|
if repo_dir is None:
|
||||||
|
self.send_error(404)
|
||||||
|
return
|
||||||
|
hook_path = os.environ.get(
|
||||||
|
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
|
||||||
|
)
|
||||||
|
peer = self.client_address[0]
|
||||||
|
hook = subprocess.run(
|
||||||
|
[hook_path, "upload-pack", str(repo_dir), peer, peer],
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if hook.returncode != 0:
|
||||||
|
detail = (hook.stderr or hook.stdout).decode(
|
||||||
|
"utf-8", errors="replace",
|
||||||
|
).rstrip()
|
||||||
|
if detail:
|
||||||
|
for line in detail.splitlines():
|
||||||
|
self.log_message("access-hook denied %s: %s",
|
||||||
|
parsed.path, line)
|
||||||
|
else:
|
||||||
|
self.log_message(
|
||||||
|
"access-hook denied %s: exit=%d (no output)",
|
||||||
|
parsed.path, hook.returncode,
|
||||||
|
)
|
||||||
|
self.send_response(403)
|
||||||
|
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(hook.stderr or hook.stdout)
|
||||||
|
return
|
||||||
|
env = os.environ.copy()
|
||||||
|
env.update({
|
||||||
|
"GIT_PROJECT_ROOT": os.environ.get("GIT_PROJECT_ROOT", "/git"),
|
||||||
|
"GIT_HTTP_EXPORT_ALL": "1",
|
||||||
|
"REQUEST_METHOD": self.command,
|
||||||
|
"PATH_INFO": parsed.path,
|
||||||
|
"QUERY_STRING": parsed.query,
|
||||||
|
"CONTENT_TYPE": self.headers.get("content-type", ""),
|
||||||
|
"CONTENT_LENGTH": self.headers.get("content-length", "0"),
|
||||||
|
"REMOTE_ADDR": self.client_address[0],
|
||||||
|
"REMOTE_PORT": str(self.client_address[1]),
|
||||||
|
"REMOTE_USER": "",
|
||||||
|
"SERVER_NAME": self.server.server_name,
|
||||||
|
"SERVER_PORT": str(self.server.server_port),
|
||||||
|
"SERVER_PROTOCOL": self.request_version,
|
||||||
|
})
|
||||||
|
for header, variable in (
|
||||||
|
("accept", "HTTP_ACCEPT"),
|
||||||
|
("content-encoding", "HTTP_CONTENT_ENCODING"),
|
||||||
|
("git-protocol", "HTTP_GIT_PROTOCOL"),
|
||||||
|
("user-agent", "HTTP_USER_AGENT"),
|
||||||
|
):
|
||||||
|
value = self.headers.get(header)
|
||||||
|
if value:
|
||||||
|
env[variable] = value
|
||||||
|
raw_length = self.headers.get("content-length", "0") or "0"
|
||||||
|
try:
|
||||||
|
length = int(raw_length)
|
||||||
|
except ValueError:
|
||||||
|
self.send_error(400, "Bad Content-Length")
|
||||||
|
return
|
||||||
|
if length < 0:
|
||||||
|
self.send_error(400, "Negative Content-Length")
|
||||||
|
return
|
||||||
|
if length > MAX_BODY_BYTES:
|
||||||
|
self.send_error(413, "Request body too large")
|
||||||
|
return
|
||||||
|
body = self.rfile.read(length) if length else b""
|
||||||
|
proc = subprocess.run(
|
||||||
|
["git", "http-backend"],
|
||||||
|
input=body,
|
||||||
|
env=env,
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
self._write_cgi_response(proc.stdout)
|
||||||
|
|
||||||
|
def _repo_dir(self, path: str) -> Path | None:
|
||||||
|
root = Path(os.environ.get("GIT_PROJECT_ROOT", "/git")).resolve()
|
||||||
|
relative = path.lstrip("/").split(".git", 1)[0] + ".git"
|
||||||
|
candidate = (root / relative).resolve()
|
||||||
|
if root not in (candidate, *candidate.parents):
|
||||||
|
return None
|
||||||
|
if not candidate.is_dir():
|
||||||
|
return None
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _is_upload_pack(path: str, query: str) -> bool:
|
||||||
|
if path.endswith("/git-upload-pack"):
|
||||||
|
return True
|
||||||
|
if path.endswith("/info/refs"):
|
||||||
|
return any(
|
||||||
|
pair == "service=git-upload-pack"
|
||||||
|
for pair in query.split("&")
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _write_cgi_response(self, raw: bytes) -> None:
|
||||||
|
head, sep, body = raw.partition(b"\r\n\r\n")
|
||||||
|
line_sep = b"\r\n"
|
||||||
|
if not sep:
|
||||||
|
head, sep, body = raw.partition(b"\n\n")
|
||||||
|
line_sep = b"\n"
|
||||||
|
status = 200
|
||||||
|
headers: list[tuple[str, str]] = []
|
||||||
|
for line in head.split(line_sep):
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
key, _, value = line.decode("latin1").partition(":")
|
||||||
|
value = value.strip()
|
||||||
|
if key.lower() == "status":
|
||||||
|
status = int(value.split()[0])
|
||||||
|
else:
|
||||||
|
headers.append((key, value))
|
||||||
|
self.send_response(status)
|
||||||
|
for key, value in headers:
|
||||||
|
self.send_header(key, value)
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, fmt: str, *args: object) -> None:
|
||||||
|
sys.stdout.write(fmt % args + "\n")
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
|
||||||
|
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
|
||||||
|
sys.stdout.write(f"git-http listening on 0.0.0.0:{port}\n")
|
||||||
|
sys.stdout.flush()
|
||||||
|
server.serve_forever()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
+15
-3
@@ -14,11 +14,23 @@ def warn(msg: str) -> None:
|
|||||||
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
|
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
|
||||||
|
|
||||||
|
|
||||||
|
def error(msg: str) -> None:
|
||||||
|
print(f"bot-bottle: error: {msg}", file=sys.stderr)
|
||||||
|
|
||||||
|
|
||||||
class Die(SystemExit):
|
class Die(SystemExit):
|
||||||
"""Raised by die() so callers (and tests) can distinguish a deliberate
|
"""Raised by die() so callers (and tests) can distinguish a deliberate
|
||||||
fatal exit from an unrelated SystemExit."""
|
fatal exit from an unrelated SystemExit.
|
||||||
|
|
||||||
|
Carries the human-facing message so a caller that suppressed stderr
|
||||||
|
— e.g. the curses dashboard, whose alternate screen is wiped when the
|
||||||
|
terminal is restored — can re-surface the reason after the fact."""
|
||||||
|
|
||||||
|
def __init__(self, code: int = 1, message: str = "") -> None:
|
||||||
|
super().__init__(code)
|
||||||
|
self.message = message
|
||||||
|
|
||||||
|
|
||||||
def die(msg: str) -> NoReturn:
|
def die(msg: str) -> NoReturn:
|
||||||
print(f"bot-bottle: error: {msg}", file=sys.stderr)
|
error(msg)
|
||||||
raise Die(1)
|
raise Die(1, msg)
|
||||||
|
|||||||
+84
-1073
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,166 @@
|
|||||||
|
"""Agent configuration manifest dataclasses."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
|
from .agent_provider import PROVIDER_TEMPLATES
|
||||||
|
from .manifest_util import ManifestError, as_json_object
|
||||||
|
from .manifest_git import GitUser
|
||||||
|
from .manifest_schema import AGENT_MODEL_KEYS
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentProvider:
|
||||||
|
"""Provider/template for the agent process inside a bottle.
|
||||||
|
|
||||||
|
`template` selects a built-in launch/runtime contract. `dockerfile`
|
||||||
|
optionally points at a custom agent-image Dockerfile while leaving
|
||||||
|
bot-bottle's sidecar infrastructure intact.
|
||||||
|
|
||||||
|
`auth_token` names the host env var that holds the provider's OAuth
|
||||||
|
token (Claude only). The provisioner injects a provider-owned egress
|
||||||
|
route for api.anthropic.com that re-injects this token as the Bearer
|
||||||
|
header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent
|
||||||
|
so the Claude Code CLI starts.
|
||||||
|
|
||||||
|
`forward_host_credentials` forwards the host Codex auth token into
|
||||||
|
the egress sidecar (Codex only).
|
||||||
|
"""
|
||||||
|
|
||||||
|
template: str = "claude"
|
||||||
|
dockerfile: str = ""
|
||||||
|
auth_token: str = ""
|
||||||
|
forward_host_credentials: bool = False
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
|
||||||
|
for k in d:
|
||||||
|
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
|
||||||
|
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
|
||||||
|
)
|
||||||
|
template = d.get("template", "claude")
|
||||||
|
if not isinstance(template, str) or not template:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.template must be a "
|
||||||
|
f"non-empty string"
|
||||||
|
)
|
||||||
|
if template not in PROVIDER_TEMPLATES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.template {template!r} "
|
||||||
|
f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
|
||||||
|
)
|
||||||
|
dockerfile = d.get("dockerfile", "")
|
||||||
|
if not isinstance(dockerfile, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.dockerfile must be a "
|
||||||
|
f"string (was {type(dockerfile).__name__})"
|
||||||
|
)
|
||||||
|
auth_token = d.get("auth_token", "")
|
||||||
|
if not isinstance(auth_token, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
|
||||||
|
f"string (was {type(auth_token).__name__})"
|
||||||
|
)
|
||||||
|
if auth_token and template != "claude":
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
||||||
|
f"supported for template 'claude'"
|
||||||
|
)
|
||||||
|
forward_host_credentials = d.get("forward_host_credentials", False)
|
||||||
|
if not isinstance(forward_host_credentials, bool):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||||
|
f"must be a boolean (was {type(forward_host_credentials).__name__})"
|
||||||
|
)
|
||||||
|
if forward_host_credentials and template != "codex":
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||||
|
"is currently only supported for template 'codex'"
|
||||||
|
)
|
||||||
|
return cls(
|
||||||
|
template=template,
|
||||||
|
dockerfile=dockerfile,
|
||||||
|
auth_token=auth_token,
|
||||||
|
forward_host_credentials=forward_host_credentials,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Agent:
|
||||||
|
bottle: str
|
||||||
|
skills: tuple[str, ...] = ()
|
||||||
|
prompt: str = ""
|
||||||
|
# Per-agent git identity (issue #94). Overlays the referenced
|
||||||
|
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
||||||
|
# `user` is allowed at the agent level; `repos` stays bottle-only
|
||||||
|
# because it carries credentials and host trust.
|
||||||
|
git_user: GitUser = GitUser()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
|
||||||
|
d = as_json_object(raw, f"agent '{name}'")
|
||||||
|
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
||||||
|
if unknown:
|
||||||
|
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
|
||||||
|
raise ManifestError(
|
||||||
|
f"agent '{name}' has unknown key(s) {sorted(unknown)}; "
|
||||||
|
f"allowed keys are {allowed}."
|
||||||
|
)
|
||||||
|
|
||||||
|
bottle = d.get("bottle")
|
||||||
|
if not isinstance(bottle, str) or not bottle:
|
||||||
|
raise ManifestError(f"agent '{name}' must declare a 'bottle' field naming a defined bottle")
|
||||||
|
if bottle not in bottle_names:
|
||||||
|
available = ", ".join(sorted(bottle_names)) or "(none defined)"
|
||||||
|
raise ManifestError(
|
||||||
|
f"agent '{name}' references bottle '{bottle}', which is not defined. "
|
||||||
|
f"Available: {available}"
|
||||||
|
)
|
||||||
|
|
||||||
|
skills: tuple[str, ...] = ()
|
||||||
|
skills_raw = d.get("skills")
|
||||||
|
if skills_raw is not None:
|
||||||
|
if not isinstance(skills_raw, list):
|
||||||
|
raise ManifestError(f"agent '{name}' skills must be an array (was {type(skills_raw).__name__})")
|
||||||
|
collected: list[str] = []
|
||||||
|
skills_list = cast(list[object], skills_raw)
|
||||||
|
for i, skill in enumerate(skills_list):
|
||||||
|
if not isinstance(skill, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"agent '{name}' skills[{i}] must be a string "
|
||||||
|
f"(was {type(skill).__name__})"
|
||||||
|
)
|
||||||
|
collected.append(skill)
|
||||||
|
skills = tuple(collected)
|
||||||
|
|
||||||
|
prompt_raw = d.get("prompt")
|
||||||
|
if prompt_raw is None:
|
||||||
|
prompt = ""
|
||||||
|
elif isinstance(prompt_raw, str):
|
||||||
|
prompt = prompt_raw
|
||||||
|
else:
|
||||||
|
raise ManifestError(f"agent '{name}' prompt must be a string (was {type(prompt_raw).__name__})")
|
||||||
|
|
||||||
|
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||||
|
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||||
|
git_user = GitUser()
|
||||||
|
git_raw = d.get("git-gate")
|
||||||
|
if git_raw is not None:
|
||||||
|
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||||
|
for k in gd.keys():
|
||||||
|
if k != "user":
|
||||||
|
raise ManifestError(
|
||||||
|
f"agent '{name}' git-gate.{k} is not allowed at the "
|
||||||
|
f"agent level; only git-gate.user (name/email) may be "
|
||||||
|
f"set on an agent. git-gate.repos is bottle-only "
|
||||||
|
f"(it carries credentials and host trust)."
|
||||||
|
)
|
||||||
|
if "user" in gd:
|
||||||
|
git_user = GitUser.from_dict(name, gd["user"])
|
||||||
|
|
||||||
|
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
"""Egress routing manifest dataclasses and helpers."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ipaddress
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
|
from .manifest_util import ManifestError, as_json_object
|
||||||
|
|
||||||
|
|
||||||
|
# Auth schemes for the egress route's optional `auth` block.
|
||||||
|
# Same values cred-proxy accepts today; `token` sidesteps the Gitea
|
||||||
|
# token-not-Bearer quirk (go-gitea/gitea#16734).
|
||||||
|
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_egress_routes(
|
||||||
|
bottle_name: str,
|
||||||
|
routes: tuple[EgressRoute, ...],
|
||||||
|
) -> None:
|
||||||
|
"""Cross-validation for `bottle.egress.routes`: hosts must be unique.
|
||||||
|
|
||||||
|
The proxy matches by exact-host (v1); duplicate hosts leave the
|
||||||
|
route choice ambiguous so we reject them up front.
|
||||||
|
|
||||||
|
No cross-validation against `bottle.git-gate.repos` is performed.
|
||||||
|
git-gate (SSH push/fetch) and egress (HTTPS) broker different
|
||||||
|
protocols; declaring both for the same host is a legitimate dev
|
||||||
|
setup."""
|
||||||
|
seen_hosts: dict[str, None] = {}
|
||||||
|
for r in routes:
|
||||||
|
key = r.Host.lower()
|
||||||
|
if key in seen_hosts:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' egress.routes has duplicate host "
|
||||||
|
f"{r.Host!r}; each host must be unique on the proxy."
|
||||||
|
)
|
||||||
|
seen_hosts[key] = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PipelockRoutePolicy:
|
||||||
|
"""Per-route pipelock policy overrides.
|
||||||
|
|
||||||
|
`TlsPassthrough` adds the route host to pipelock's
|
||||||
|
`tls_interception.passthrough_domains`, so pipelock still enforces
|
||||||
|
the hostname allowlist but does not MITM/decrypt request bodies or
|
||||||
|
headers for that host.
|
||||||
|
|
||||||
|
`SsrfIpAllowlist` adds explicit IPs/CIDRs to pipelock's SSRF
|
||||||
|
allowlist for private/internal destinations behind this route.
|
||||||
|
"""
|
||||||
|
|
||||||
|
TlsPassthrough: bool = False
|
||||||
|
SsrfIpAllowlist: tuple[str, ...] = ()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(
|
||||||
|
cls, bottle_name: str, idx: int, raw: object,
|
||||||
|
) -> "PipelockRoutePolicy":
|
||||||
|
label = f"bottle '{bottle_name}' egress.routes[{idx}] pipelock"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
for k in d:
|
||||||
|
if k not in ("tls_passthrough", "ssrf_ip_allowlist"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} has unknown key {k!r}; "
|
||||||
|
f"only 'tls_passthrough' and 'ssrf_ip_allowlist' "
|
||||||
|
f"are accepted"
|
||||||
|
)
|
||||||
|
tls_passthrough_raw = d.get("tls_passthrough", False)
|
||||||
|
if not isinstance(tls_passthrough_raw, bool):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label}.tls_passthrough must be a boolean "
|
||||||
|
f"(was {type(tls_passthrough_raw).__name__})"
|
||||||
|
)
|
||||||
|
ssrf_raw = d.get("ssrf_ip_allowlist", [])
|
||||||
|
if not isinstance(ssrf_raw, list):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label}.ssrf_ip_allowlist must be an array "
|
||||||
|
f"(was {type(ssrf_raw).__name__})"
|
||||||
|
)
|
||||||
|
ssrf_ip_allowlist: list[str] = []
|
||||||
|
for j, item in enumerate(ssrf_raw):
|
||||||
|
if not isinstance(item, str) or not item:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label}.ssrf_ip_allowlist[{j}] must be a non-empty "
|
||||||
|
f"string (was {type(item).__name__})"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
ipaddress.ip_network(item, strict=False)
|
||||||
|
except ValueError as e:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label}.ssrf_ip_allowlist[{j}] must be an IP address "
|
||||||
|
f"or CIDR (was {item!r}): {e}"
|
||||||
|
)
|
||||||
|
ssrf_ip_allowlist.append(item)
|
||||||
|
return cls(
|
||||||
|
TlsPassthrough=tls_passthrough_raw,
|
||||||
|
SsrfIpAllowlist=tuple(ssrf_ip_allowlist),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class EgressRoute:
|
||||||
|
"""One route on the per-bottle egress sidecar (PRD 0017).
|
||||||
|
|
||||||
|
`Host` matches the request's hostname (case-insensitive). The
|
||||||
|
optional `PathAllowlist` constrains the URL path to a set of
|
||||||
|
prefixes; empty tuple means no path-level filtering. The optional
|
||||||
|
`AuthScheme` / `TokenRef` pair drives credential injection:
|
||||||
|
when set, the proxy strips any inbound Authorization and injects
|
||||||
|
`<AuthScheme> <value-of-host-env-named-by-TokenRef>`. When the
|
||||||
|
manifest's `auth` block is omitted both fields are empty strings —
|
||||||
|
no Authorization is written, no token forwarded.
|
||||||
|
|
||||||
|
`Role` is reserved for future use; all role strings are currently
|
||||||
|
rejected by the validator.
|
||||||
|
|
||||||
|
Validation rules (enforced in `from_dict`):
|
||||||
|
- `host` required, non-empty.
|
||||||
|
- `path_allowlist` optional, list of absolute path prefixes.
|
||||||
|
- `auth` optional. If present, MUST carry both `scheme` and
|
||||||
|
`token_ref` as non-empty strings; an empty `auth: {}` is an
|
||||||
|
error rather than a synonym for "no auth" (omit `auth` for
|
||||||
|
that case).
|
||||||
|
- `role` optional, reserved — any non-empty value is rejected.
|
||||||
|
"""
|
||||||
|
|
||||||
|
Host: str
|
||||||
|
PathAllowlist: tuple[str, ...] = ()
|
||||||
|
AuthScheme: str = ""
|
||||||
|
TokenRef: str = ""
|
||||||
|
Role: tuple[str, ...] = ()
|
||||||
|
Pipelock: PipelockRoutePolicy = field(default_factory=PipelockRoutePolicy)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
||||||
|
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
host = d.get("host")
|
||||||
|
if not isinstance(host, str) or not host:
|
||||||
|
raise ManifestError(f"{label} missing required string field 'host'")
|
||||||
|
|
||||||
|
path_allow_raw = d.get("path_allowlist")
|
||||||
|
prefixes: tuple[str, ...] = ()
|
||||||
|
if path_allow_raw is not None:
|
||||||
|
if not isinstance(path_allow_raw, list):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} path_allowlist must be an array "
|
||||||
|
f"(was {type(path_allow_raw).__name__})"
|
||||||
|
)
|
||||||
|
path_list = cast(list[object], path_allow_raw)
|
||||||
|
collected: list[str] = []
|
||||||
|
for j, p in enumerate(path_list):
|
||||||
|
if not isinstance(p, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} path_allowlist[{j}] must be a string "
|
||||||
|
f"(was {type(p).__name__})"
|
||||||
|
)
|
||||||
|
if not p.startswith("/"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} path_allowlist[{j}] {p!r} must be an "
|
||||||
|
f"absolute path prefix starting with '/'"
|
||||||
|
)
|
||||||
|
collected.append(p)
|
||||||
|
prefixes = tuple(collected)
|
||||||
|
|
||||||
|
auth_scheme = ""
|
||||||
|
token_ref = ""
|
||||||
|
if "auth" in d:
|
||||||
|
auth_raw = d.get("auth")
|
||||||
|
auth_d = as_json_object(auth_raw, f"{label} auth")
|
||||||
|
if not auth_d:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} auth is empty ({{}}); omit the 'auth' key "
|
||||||
|
f"entirely if this route is unauthenticated. Otherwise "
|
||||||
|
f"both 'scheme' and 'token_ref' are required."
|
||||||
|
)
|
||||||
|
auth_scheme_raw = auth_d.get("scheme")
|
||||||
|
if not isinstance(auth_scheme_raw, str) or not auth_scheme_raw:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} auth.scheme is required when 'auth' is set "
|
||||||
|
f"(non-empty string)"
|
||||||
|
)
|
||||||
|
if auth_scheme_raw not in EGRESS_AUTH_SCHEMES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} auth.scheme {auth_scheme_raw!r} is not one of "
|
||||||
|
f"{', '.join(EGRESS_AUTH_SCHEMES)}"
|
||||||
|
)
|
||||||
|
token_ref_raw = auth_d.get("token_ref")
|
||||||
|
if not isinstance(token_ref_raw, str) or not token_ref_raw:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} auth.token_ref is required when 'auth' is set "
|
||||||
|
f"(name of the host env var holding the token value)"
|
||||||
|
)
|
||||||
|
for k in auth_d:
|
||||||
|
if k not in ("scheme", "token_ref"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} auth has unknown key {k!r}; "
|
||||||
|
f"only 'scheme' and 'token_ref' are accepted"
|
||||||
|
)
|
||||||
|
auth_scheme = auth_scheme_raw
|
||||||
|
token_ref = token_ref_raw
|
||||||
|
|
||||||
|
role_raw = d.get("role")
|
||||||
|
roles: tuple[str, ...] = ()
|
||||||
|
if role_raw is None:
|
||||||
|
roles = ()
|
||||||
|
elif isinstance(role_raw, str):
|
||||||
|
roles = (role_raw,)
|
||||||
|
elif isinstance(role_raw, list):
|
||||||
|
role_list = cast(list[object], role_raw)
|
||||||
|
collected_roles: list[str] = []
|
||||||
|
for r in role_list:
|
||||||
|
if not isinstance(r, str):
|
||||||
|
raise ManifestError(f"{label} role items must be strings (got {type(r).__name__})")
|
||||||
|
collected_roles.append(r)
|
||||||
|
roles = tuple(collected_roles)
|
||||||
|
else:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} role must be a string or a list of strings "
|
||||||
|
f"(was {type(role_raw).__name__})"
|
||||||
|
)
|
||||||
|
if roles:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} role {roles[0]!r} is not accepted; "
|
||||||
|
f"the 'role' field is reserved for future use"
|
||||||
|
)
|
||||||
|
|
||||||
|
pipelock = (
|
||||||
|
PipelockRoutePolicy.from_dict(bottle_name, idx, d["pipelock"])
|
||||||
|
if "pipelock" in d
|
||||||
|
else PipelockRoutePolicy()
|
||||||
|
)
|
||||||
|
|
||||||
|
for k in d:
|
||||||
|
if k not in ("host", "path_allowlist", "auth", "role", "pipelock"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} has unknown key {k!r}; accepted keys are "
|
||||||
|
f"'host', 'path_allowlist', 'auth', 'role', 'pipelock'"
|
||||||
|
)
|
||||||
|
|
||||||
|
return cls(
|
||||||
|
Host=host,
|
||||||
|
PathAllowlist=prefixes,
|
||||||
|
AuthScheme=auth_scheme,
|
||||||
|
TokenRef=token_ref,
|
||||||
|
Role=roles,
|
||||||
|
Pipelock=pipelock,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class EgressConfig:
|
||||||
|
"""Per-bottle egress configuration. Today this is just the
|
||||||
|
route table; the nesting under `egress:` leaves room for
|
||||||
|
per-bottle proxy settings (port override, log level, etc.) in
|
||||||
|
follow-ups."""
|
||||||
|
|
||||||
|
routes: tuple[EgressRoute, ...] = ()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
|
||||||
|
routes_raw = d.get("routes")
|
||||||
|
routes: tuple[EgressRoute, ...] = ()
|
||||||
|
if routes_raw is not None:
|
||||||
|
if not isinstance(routes_raw, list):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' egress.routes must be an array "
|
||||||
|
f"(was {type(routes_raw).__name__})"
|
||||||
|
)
|
||||||
|
routes_list = cast(list[object], routes_raw)
|
||||||
|
routes = tuple(
|
||||||
|
EgressRoute.from_dict(bottle_name, i, entry)
|
||||||
|
for i, entry in enumerate(routes_list)
|
||||||
|
)
|
||||||
|
validate_egress_routes(bottle_name, routes)
|
||||||
|
for k in d:
|
||||||
|
if k != "routes":
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' egress has unknown key {k!r}; "
|
||||||
|
f"only 'routes' is accepted"
|
||||||
|
)
|
||||||
|
return cls(routes=routes)
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
"""Internal bottle `extends:` resolution for manifests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from .manifest import Bottle, GitEntry
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
|
||||||
|
"""Apply `extends:` chains and return resolved Bottle objects."""
|
||||||
|
cache: dict[str, Bottle] = {}
|
||||||
|
for name in raws:
|
||||||
|
if name not in cache:
|
||||||
|
_resolve_one_bottle(name, raws, cache, ())
|
||||||
|
return cache
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_one_bottle(
|
||||||
|
name: str,
|
||||||
|
raws: dict[str, dict[str, object]],
|
||||||
|
cache: dict[str, Bottle],
|
||||||
|
seen: tuple[str, ...],
|
||||||
|
) -> Bottle:
|
||||||
|
from .manifest import Bottle, ManifestError
|
||||||
|
|
||||||
|
if name in cache:
|
||||||
|
return cache[name]
|
||||||
|
if name in seen:
|
||||||
|
chain = " -> ".join(seen + (name,))
|
||||||
|
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
|
||||||
|
raw = raws[name]
|
||||||
|
parent_name_raw = raw.get("extends")
|
||||||
|
# Strip `extends:` before passing to Bottle.from_dict so it
|
||||||
|
# is not accidentally treated as a real Bottle field by future
|
||||||
|
# schema additions. It is only meaningful here.
|
||||||
|
child_raw = {k: v for k, v in raw.items() if k != "extends"}
|
||||||
|
|
||||||
|
if parent_name_raw is None:
|
||||||
|
bottle = Bottle.from_dict(name, child_raw)
|
||||||
|
cache[name] = bottle
|
||||||
|
return bottle
|
||||||
|
|
||||||
|
if not isinstance(parent_name_raw, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{name}' extends must be a string "
|
||||||
|
f"(was {type(parent_name_raw).__name__})"
|
||||||
|
)
|
||||||
|
parent_name: str = parent_name_raw
|
||||||
|
if parent_name == name:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{name}' extends itself; remove the "
|
||||||
|
f"self-reference"
|
||||||
|
)
|
||||||
|
if parent_name not in raws:
|
||||||
|
avail = ", ".join(sorted(raws.keys())) or "(none)"
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{name}' extends '{parent_name}' which is not "
|
||||||
|
f"defined. Available bottles: {avail}"
|
||||||
|
)
|
||||||
|
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
|
||||||
|
bottle = _merge_bottles(parent, child_raw, name)
|
||||||
|
cache[name] = bottle
|
||||||
|
return bottle
|
||||||
|
|
||||||
|
|
||||||
|
def _merge_bottles(
|
||||||
|
parent: Bottle,
|
||||||
|
child_raw: dict[str, object],
|
||||||
|
name: str,
|
||||||
|
) -> Bottle:
|
||||||
|
"""Apply PRD 0025 merge rules."""
|
||||||
|
from .manifest import Bottle, GitUser
|
||||||
|
from .manifest_egress import validate_egress_routes
|
||||||
|
|
||||||
|
# Parse the child's declared fields into a Bottle (with the
|
||||||
|
# usual defaults for anything missing). Validation runs the same
|
||||||
|
# way it would for a leaf bottle: typos / wrong types die here.
|
||||||
|
child = Bottle.from_dict(name, child_raw)
|
||||||
|
|
||||||
|
# env: dict merge, child wins on collision.
|
||||||
|
merged_env = {**parent.env, **child.env}
|
||||||
|
|
||||||
|
# git-gate.user: per-field overlay. Each non-empty field on child
|
||||||
|
# wins; empties fall through to parent. The default GitUser()
|
||||||
|
# is two empty strings, so a child that omits git-gate.user
|
||||||
|
# inherits the parent's user verbatim.
|
||||||
|
merged_git_user = GitUser(
|
||||||
|
name=child.git_user.name or parent.git_user.name,
|
||||||
|
email=child.git_user.email or parent.git_user.email,
|
||||||
|
)
|
||||||
|
|
||||||
|
# git-gate.repos: missing means inherit; an explicit empty object
|
||||||
|
# clears; otherwise parent and child merge by UpstreamHost with
|
||||||
|
# child entries replacing duplicate hosts.
|
||||||
|
if _child_declares_git_gate_repos(child_raw):
|
||||||
|
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
|
||||||
|
else:
|
||||||
|
merged_git = parent.git
|
||||||
|
|
||||||
|
# Presence-driven full-replace for the remaining list-valued +
|
||||||
|
# scalar fields.
|
||||||
|
merged_egress = child.egress if "egress" in child_raw else parent.egress
|
||||||
|
merged_agent_provider = (
|
||||||
|
child.agent_provider
|
||||||
|
if "agent_provider" in child_raw
|
||||||
|
else parent.agent_provider
|
||||||
|
)
|
||||||
|
merged_supervise = (
|
||||||
|
child.supervise if "supervise" in child_raw else parent.supervise
|
||||||
|
)
|
||||||
|
validate_egress_routes(name, merged_egress.routes)
|
||||||
|
|
||||||
|
return Bottle(
|
||||||
|
env=merged_env,
|
||||||
|
agent_provider=merged_agent_provider,
|
||||||
|
git=merged_git,
|
||||||
|
git_user=merged_git_user,
|
||||||
|
egress=merged_egress,
|
||||||
|
supervise=merged_supervise,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
|
||||||
|
from .manifest_util import as_json_object
|
||||||
|
|
||||||
|
git_raw = child_raw.get("git-gate")
|
||||||
|
if git_raw is None:
|
||||||
|
return False
|
||||||
|
git_obj = as_json_object(git_raw, "child git-gate")
|
||||||
|
return "repos" in git_obj
|
||||||
|
|
||||||
|
|
||||||
|
def _merge_git_remotes(
|
||||||
|
parent: tuple[GitEntry, ...],
|
||||||
|
child: tuple[GitEntry, ...],
|
||||||
|
) -> tuple[GitEntry, ...]:
|
||||||
|
by_host = {entry.UpstreamHost: entry for entry in parent}
|
||||||
|
for entry in child:
|
||||||
|
by_host[entry.UpstreamHost] = entry
|
||||||
|
return tuple(by_host.values())
|
||||||
@@ -0,0 +1,222 @@
|
|||||||
|
"""Git-related manifest dataclasses and helpers."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from .manifest_util import ManifestError, as_json_object
|
||||||
|
|
||||||
|
# Shell-safe characters for git-gate repo names. Names are embedded in
|
||||||
|
# the generated entrypoint shell script (shlex.quote is the primary
|
||||||
|
# defence; this regex is belt-and-suspenders and documents intent).
|
||||||
|
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
|
||||||
|
|
||||||
|
|
||||||
|
def _opt_str(value: object, label: str) -> str:
|
||||||
|
if value is None:
|
||||||
|
return ""
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise ManifestError(f"{label} must be a string (was {type(value).__name__})")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
|
||||||
|
"""Parse `ssh://user@host[:port]/path` into (user, host, port, path).
|
||||||
|
Dies if `url` doesn't match the ssh:// shape v1 supports. Default
|
||||||
|
port is 22 (matches OpenSSH)."""
|
||||||
|
if not url.startswith("ssh://"):
|
||||||
|
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
|
||||||
|
rest = url[len("ssh://"):]
|
||||||
|
if "@" not in rest:
|
||||||
|
raise ManifestError(f"{label} must include a user (e.g. ssh://git@host/path.git); was {url!r}")
|
||||||
|
user, _, hostpart = rest.partition("@")
|
||||||
|
if not user:
|
||||||
|
raise ManifestError(f"{label} user is empty in {url!r}")
|
||||||
|
if "/" not in hostpart:
|
||||||
|
raise ManifestError(f"{label} must include a path (e.g. ssh://git@host/path.git); was {url!r}")
|
||||||
|
hostport, _, path = hostpart.partition("/")
|
||||||
|
if not path:
|
||||||
|
raise ManifestError(f"{label} path is empty in {url!r}")
|
||||||
|
if ":" in hostport:
|
||||||
|
host, _, port = hostport.partition(":")
|
||||||
|
if not port.isdigit():
|
||||||
|
raise ManifestError(f"{label} port must be numeric in {url!r}")
|
||||||
|
else:
|
||||||
|
host = hostport
|
||||||
|
port = "22"
|
||||||
|
if not host:
|
||||||
|
raise ManifestError(f"{label} host is empty in {url!r}")
|
||||||
|
return (user, host, port, path)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
|
||||||
|
seen: dict[str, None] = {}
|
||||||
|
for g in git:
|
||||||
|
if g.Name in seen:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.repos has duplicate name '{g.Name}'; "
|
||||||
|
f"each entry maps to a distinct bare repo on the gate."
|
||||||
|
)
|
||||||
|
seen[g.Name] = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitEntry:
|
||||||
|
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
||||||
|
talk to. `Upstream` is the real remote URL the agent would push to
|
||||||
|
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
|
||||||
|
and `IdentityFile` is the SSH key the gate uses to push that repo
|
||||||
|
upstream after gitleaks passes. The agent itself never holds the
|
||||||
|
upstream credential.
|
||||||
|
|
||||||
|
The Upstream URL is parsed once at construction and the pieces are
|
||||||
|
stashed in the `Upstream*` fields so the git-gate render step
|
||||||
|
doesn't have to re-parse.
|
||||||
|
|
||||||
|
Manifest source: `git-gate.repos.<Name>` (PRD 0047). The YAML keys
|
||||||
|
are `url`, `identity`, and `host_key`; the internal field names are
|
||||||
|
stable across that rename."""
|
||||||
|
|
||||||
|
Name: str
|
||||||
|
Upstream: str
|
||||||
|
IdentityFile: str
|
||||||
|
KnownHostKey: str = ""
|
||||||
|
RemoteKey: str = ""
|
||||||
|
UpstreamUser: str = ""
|
||||||
|
UpstreamHost: str = ""
|
||||||
|
UpstreamPort: str = ""
|
||||||
|
UpstreamPath: str = ""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_repos_entry(
|
||||||
|
cls, bottle_name: str, repo_name: str, raw: object
|
||||||
|
) -> "GitEntry":
|
||||||
|
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
||||||
|
|
||||||
|
YAML keys: `url` (required), `identity` (required),
|
||||||
|
`host_key` (optional). The repo_name becomes `Name`."""
|
||||||
|
if not repo_name:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.repos has an empty key"
|
||||||
|
)
|
||||||
|
if not _GIT_NAME_RE.match(repo_name):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.repos name {repo_name!r} is invalid; "
|
||||||
|
f"allowed characters: A-Z a-z 0-9 . _ -"
|
||||||
|
)
|
||||||
|
label = f"git-gate.repos[{repo_name!r}]"
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||||
|
for k in d:
|
||||||
|
if k not in {"url", "identity", "host_key"}:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||||
|
f"allowed: url, identity, host_key"
|
||||||
|
)
|
||||||
|
upstream = d.get("url")
|
||||||
|
if not isinstance(upstream, str) or not upstream:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||||
|
)
|
||||||
|
ident = d.get("identity")
|
||||||
|
if not isinstance(ident, str) or not ident:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label} missing required string field 'identity'"
|
||||||
|
)
|
||||||
|
khk = _opt_str(
|
||||||
|
d.get("host_key"),
|
||||||
|
f"bottle '{bottle_name}' {label} host_key",
|
||||||
|
)
|
||||||
|
user, host, port, path = parse_git_upstream(
|
||||||
|
upstream, f"bottle '{bottle_name}' {label} url"
|
||||||
|
)
|
||||||
|
return cls(
|
||||||
|
Name=repo_name,
|
||||||
|
Upstream=upstream,
|
||||||
|
IdentityFile=ident,
|
||||||
|
KnownHostKey=khk,
|
||||||
|
RemoteKey=host,
|
||||||
|
UpstreamUser=user,
|
||||||
|
UpstreamHost=host,
|
||||||
|
UpstreamPort=port,
|
||||||
|
UpstreamPath=path,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitUser:
|
||||||
|
"""Per-bottle `git config --global user.name` / `user.email`
|
||||||
|
pair (issue #86). The agent's commits inside the bottle are
|
||||||
|
attributed to this identity rather than the agent image's
|
||||||
|
image-baked default (no user, or whatever the image dropped
|
||||||
|
in). Either or both fields can be set independently.
|
||||||
|
|
||||||
|
`from_dict` is forgiving on shape (a single missing field is
|
||||||
|
fine — we just skip that config line at provisioning) but
|
||||||
|
strict on types (string-or-die)."""
|
||||||
|
|
||||||
|
name: str = ""
|
||||||
|
email: str = ""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
||||||
|
for k in d.keys():
|
||||||
|
if k not in {"name", "email"}:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
|
||||||
|
f"allowed: name, email"
|
||||||
|
)
|
||||||
|
name = d.get("name", "")
|
||||||
|
email = d.get("email", "")
|
||||||
|
if not isinstance(name, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.user.name must be a string "
|
||||||
|
f"(was {type(name).__name__})"
|
||||||
|
)
|
||||||
|
if not isinstance(email, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.user.email must be a string "
|
||||||
|
f"(was {type(email).__name__})"
|
||||||
|
)
|
||||||
|
if not name and not email:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate.user is set but neither "
|
||||||
|
f"name nor email is non-empty; remove the block or "
|
||||||
|
f"fill at least one field."
|
||||||
|
)
|
||||||
|
return cls(name=name, email=email)
|
||||||
|
|
||||||
|
def is_empty(self) -> bool:
|
||||||
|
return not self.name and not self.email
|
||||||
|
|
||||||
|
|
||||||
|
def parse_git_gate_config(
|
||||||
|
bottle_name: str,
|
||||||
|
raw: object,
|
||||||
|
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||||
|
for k in d.keys():
|
||||||
|
if k not in {"user", "repos"}:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
||||||
|
f"allowed: user, repos"
|
||||||
|
)
|
||||||
|
|
||||||
|
git_user = (
|
||||||
|
GitUser.from_dict(bottle_name, d["user"])
|
||||||
|
if "user" in d
|
||||||
|
else GitUser()
|
||||||
|
)
|
||||||
|
|
||||||
|
git: tuple[GitEntry, ...] = ()
|
||||||
|
repos_raw = d.get("repos")
|
||||||
|
if repos_raw is not None:
|
||||||
|
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
|
||||||
|
git = tuple(
|
||||||
|
GitEntry.from_repos_entry(bottle_name, name, entry)
|
||||||
|
for name, entry in repos.items()
|
||||||
|
)
|
||||||
|
validate_unique_git_names(bottle_name, git)
|
||||||
|
|
||||||
|
return git, git_user
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
"""Internal per-file Markdown manifest loader."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from .log import warn
|
||||||
|
from .manifest_schema import (
|
||||||
|
entity_name_from_path,
|
||||||
|
validate_agent_frontmatter_keys,
|
||||||
|
validate_bottle_frontmatter_keys,
|
||||||
|
)
|
||||||
|
from .yaml_subset import YamlSubsetError, parse_frontmatter
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from .manifest import Agent, Bottle
|
||||||
|
|
||||||
|
|
||||||
|
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
|
||||||
|
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
|
||||||
|
not. The manifest format changed in PRD 0011 and we do not want
|
||||||
|
to silently leave the JSON content unused."""
|
||||||
|
from .manifest import ManifestError
|
||||||
|
|
||||||
|
legacy = dir_path / "bot-bottle.json"
|
||||||
|
if legacy.is_file() and not md_dir.exists():
|
||||||
|
raise ManifestError(
|
||||||
|
f"found {legacy} but {md_dir} does not exist. The manifest "
|
||||||
|
f"format changed in PRD 0011 — rewrite the JSON content "
|
||||||
|
f"as per-file Markdown under {md_dir}/bottles/ and "
|
||||||
|
f"{md_dir}/agents/. See README.md for the schema. "
|
||||||
|
f"({label})"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
|
||||||
|
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
|
||||||
|
`{name: Bottle}`. Missing dir returns an empty dict."""
|
||||||
|
from .manifest import ManifestError
|
||||||
|
from .manifest_extends import resolve_bottles
|
||||||
|
|
||||||
|
raws: dict[str, dict[str, object]] = {}
|
||||||
|
if not bottles_dir.is_dir():
|
||||||
|
return {}
|
||||||
|
for path in sorted(bottles_dir.glob("*.md")):
|
||||||
|
name = entity_name_from_path(path)
|
||||||
|
if name is None:
|
||||||
|
warn(
|
||||||
|
f"skipping {path}: filename must match "
|
||||||
|
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
fm, _body = parse_frontmatter(path.read_text())
|
||||||
|
except OSError as e:
|
||||||
|
raise ManifestError(f"could not read {path}: {e}")
|
||||||
|
except YamlSubsetError as e:
|
||||||
|
raise ManifestError(f"{path}: {e}")
|
||||||
|
validate_bottle_frontmatter_keys(path, fm.keys())
|
||||||
|
raws[name] = fm
|
||||||
|
return resolve_bottles(raws)
|
||||||
|
|
||||||
|
|
||||||
|
def load_agents_from_dir(
|
||||||
|
agents_dir: Path,
|
||||||
|
bottle_names: set[str],
|
||||||
|
*,
|
||||||
|
source: str,
|
||||||
|
) -> dict[str, Agent]:
|
||||||
|
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
||||||
|
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
||||||
|
Missing dir returns an empty dict."""
|
||||||
|
from .manifest import Agent, ManifestError
|
||||||
|
|
||||||
|
out: dict[str, Agent] = {}
|
||||||
|
if not agents_dir.is_dir():
|
||||||
|
return out
|
||||||
|
for path in sorted(agents_dir.glob("*.md")):
|
||||||
|
name = entity_name_from_path(path)
|
||||||
|
if name is None:
|
||||||
|
warn(
|
||||||
|
f"skipping {path}: filename must match "
|
||||||
|
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
fm, body = parse_frontmatter(path.read_text())
|
||||||
|
except OSError as e:
|
||||||
|
raise ManifestError(f"could not read {path}: {e}")
|
||||||
|
except YamlSubsetError as e:
|
||||||
|
raise ManifestError(f"{path}: {e}")
|
||||||
|
validate_agent_frontmatter_keys(path, fm.keys())
|
||||||
|
# Build the dict Agent.from_dict expects. The body becomes
|
||||||
|
# prompt; Claude Code passthrough fields stay in fm and get
|
||||||
|
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
|
||||||
|
agent_dict: dict[str, object] = {
|
||||||
|
"bottle": fm.get("bottle"),
|
||||||
|
"skills": fm.get("skills", []),
|
||||||
|
"prompt": body.strip(),
|
||||||
|
}
|
||||||
|
if "git-gate" in fm:
|
||||||
|
agent_dict["git-gate"] = fm["git-gate"]
|
||||||
|
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
|
||||||
|
return out
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
"""Internal manifest schema policy helpers."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Filename-as-key uses kebab-case ASCII. The first character is a
|
||||||
|
# letter so we don't conflict with hidden files / Markdown special
|
||||||
|
# names (`.md`, `_template.md`, etc.). Filenames that fail this
|
||||||
|
# pattern are skipped with a warning rather than crashing the load.
|
||||||
|
_FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
|
||||||
|
|
||||||
|
|
||||||
|
# Frontmatter keys we accept on each entity. Anything not in these
|
||||||
|
# sets dies with a "did you mean" pointer: typos should not silently
|
||||||
|
# ghost into an empty config.
|
||||||
|
BOTTLE_KEYS = frozenset(
|
||||||
|
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
|
||||||
|
)
|
||||||
|
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
|
||||||
|
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
|
||||||
|
|
||||||
|
# Claude Code subagent fields bot-bottle ignores at launch but does
|
||||||
|
# not reject. This lets the same file double as
|
||||||
|
# `~/.claude/agents/*.md` without modification.
|
||||||
|
CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS = frozenset({
|
||||||
|
"name", "description", "model", "color", "memory",
|
||||||
|
})
|
||||||
|
AGENT_KEYS = (
|
||||||
|
AGENT_KEYS_REQUIRED | AGENT_KEYS_OPTIONAL | CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS
|
||||||
|
)
|
||||||
|
AGENT_MODEL_KEYS = AGENT_KEYS | frozenset({"prompt"})
|
||||||
|
|
||||||
|
|
||||||
|
def entity_name_from_path(path: Path) -> str | None:
|
||||||
|
"""Return the entity name implied by the filename, or None if the
|
||||||
|
filename does not fit the [a-z][a-z0-9-]* convention."""
|
||||||
|
if path.suffix != ".md":
|
||||||
|
return None
|
||||||
|
stem = path.stem
|
||||||
|
if not _FILENAME_RX.match(stem):
|
||||||
|
return None
|
||||||
|
return stem
|
||||||
|
|
||||||
|
|
||||||
|
def validate_bottle_frontmatter_keys(path: Path, keys: object) -> None:
|
||||||
|
_validate_frontmatter_keys("bottle", path, keys, BOTTLE_KEYS)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_agent_frontmatter_keys(path: Path, keys: object) -> None:
|
||||||
|
_validate_frontmatter_keys("agent", path, keys, AGENT_KEYS)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_frontmatter_keys(
|
||||||
|
kind: str,
|
||||||
|
path: Path,
|
||||||
|
keys: object,
|
||||||
|
allowed_keys: frozenset[str],
|
||||||
|
) -> None:
|
||||||
|
from .manifest_util import ManifestError
|
||||||
|
|
||||||
|
key_set = set(keys)
|
||||||
|
unknown = key_set - allowed_keys
|
||||||
|
if unknown:
|
||||||
|
allowed = ", ".join(sorted(allowed_keys))
|
||||||
|
raise ManifestError(
|
||||||
|
f"{kind} file {path}: unknown frontmatter key(s) "
|
||||||
|
f"{sorted(unknown)}; allowed keys are {allowed}."
|
||||||
|
)
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
"""Shared manifest primitives used by all manifest sub-modules."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
|
|
||||||
|
class ManifestError(Exception):
|
||||||
|
"""A manifest file (or the manifest tree) is invalid."""
|
||||||
|
|
||||||
|
|
||||||
|
def as_json_object(value: object, label: str) -> dict[str, object]:
|
||||||
|
"""Assert that `value` is a JSON object (str-keyed dict) and return
|
||||||
|
a view typed as `dict[str, object]` so downstream `.get(...)` calls
|
||||||
|
have a typed surface."""
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise ManifestError(f"{label} must be a JSON object (was {type(value).__name__})")
|
||||||
|
items = cast(dict[object, object], value)
|
||||||
|
out: dict[str, object] = {}
|
||||||
|
for k, v in items.items():
|
||||||
|
if not isinstance(k, str):
|
||||||
|
raise ManifestError(f"{label} keys must be strings (found {type(k).__name__})")
|
||||||
|
out[k] = v
|
||||||
|
return out
|
||||||
+228
-30
@@ -19,9 +19,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import cast
|
|
||||||
|
|
||||||
from .egress import EGRESS_HOSTNAME, egress_routes_for_bottle
|
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
|
||||||
from .supervise import SUPERVISE_HOSTNAME
|
from .supervise import SUPERVISE_HOSTNAME
|
||||||
from .manifest import Bottle
|
from .manifest import Bottle
|
||||||
|
|
||||||
@@ -50,14 +49,17 @@ PIPELOCK_HOSTNAME = "pipelock"
|
|||||||
# --- Allowlist resolution --------------------------------------------------
|
# --- Allowlist resolution --------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def pipelock_effective_allowlist(bottle: Bottle) -> list[str]:
|
def pipelock_effective_allowlist(
|
||||||
|
bottle: Bottle,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
|
) -> list[str]:
|
||||||
"""Hostnames pipelock allows. Sorted for stability.
|
"""Hostnames pipelock allows. Sorted for stability.
|
||||||
|
|
||||||
Always mirrors `egress_routes_for_bottle(bottle)` — egress is the
|
Always mirrors `egress_routes_for_bottle(bottle, provider_routes)` —
|
||||||
single allowlist surface, and pipelock's allowlist is the downstream
|
egress is the single allowlist surface, and pipelock's allowlist is
|
||||||
copy for defense-in-depth + DLP body scanning. For bottles without
|
the downstream copy for defense-in-depth + DLP body scanning. For
|
||||||
any `egress.routes[]` declared, this is empty except for supervise
|
bottles without any `egress.routes[]` declared, this is empty except
|
||||||
sidecar traffic when `supervise: true`.
|
for supervise sidecar traffic when `supervise: true`.
|
||||||
|
|
||||||
The supervise sidecar's hostname is auto-added when supervise
|
The supervise sidecar's hostname is auto-added when supervise
|
||||||
is enabled (sibling-sidecar traffic that flows through pipelock
|
is enabled (sibling-sidecar traffic that flows through pipelock
|
||||||
@@ -65,7 +67,7 @@ def pipelock_effective_allowlist(bottle: Bottle) -> list[str]:
|
|||||||
`bottle.git` do NOT contribute here — git traffic flows
|
`bottle.git` do NOT contribute here — git traffic flows
|
||||||
through git-gate (PRD 0008), not pipelock."""
|
through git-gate (PRD 0008), not pipelock."""
|
||||||
seen: dict[str, None] = {}
|
seen: dict[str, None] = {}
|
||||||
for r in egress_routes_for_bottle(bottle):
|
for r in egress_routes_for_bottle(bottle, provider_routes):
|
||||||
if r.host:
|
if r.host:
|
||||||
seen.setdefault(r.host, None)
|
seen.setdefault(r.host, None)
|
||||||
if bottle.supervise:
|
if bottle.supervise:
|
||||||
@@ -98,19 +100,23 @@ def pipelock_seed_phrase_detection_enabled(bottle: Bottle) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def pipelock_effective_tls_passthrough(bottle: Bottle) -> list[str]:
|
def pipelock_effective_tls_passthrough(
|
||||||
|
bottle: Bottle,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
|
) -> list[str]:
|
||||||
"""Hostnames pipelock should pass through (no TLS MITM).
|
"""Hostnames pipelock should pass through (no TLS MITM).
|
||||||
|
|
||||||
A route opts in with `pipelock.tls_passthrough: true`. This is
|
A manifest route opts in with `pipelock.tls_passthrough: true`
|
||||||
useful for provider API routes where egress injects the
|
(lifted into `EgressRoute.tls_passthrough` in `egress_manifest_routes`).
|
||||||
Authorization header after the agent boundary; pipelock still
|
Provider routes that set `tls_passthrough=True` (e.g. Codex credential
|
||||||
enforces the host allowlist but does not decrypt and scan that
|
routes where egress injects the host bearer after the agent boundary)
|
||||||
provider request.
|
are also included. Both arrive via `egress_routes_for_bottle` — no
|
||||||
|
provider-specific branching needed here.
|
||||||
"""
|
"""
|
||||||
seen: dict[str, None] = {host: None for host in DEFAULT_TLS_PASSTHROUGH}
|
seen: dict[str, None] = {host: None for host in DEFAULT_TLS_PASSTHROUGH}
|
||||||
for route in bottle.egress.routes:
|
for route in egress_routes_for_bottle(bottle, provider_routes):
|
||||||
if route.Pipelock.TlsPassthrough:
|
if route.tls_passthrough:
|
||||||
seen.setdefault(route.Host, None)
|
seen.setdefault(route.host, None)
|
||||||
return sorted(seen.keys())
|
return sorted(seen.keys())
|
||||||
|
|
||||||
|
|
||||||
@@ -142,6 +148,7 @@ def pipelock_build_config(
|
|||||||
ca_cert_path: str = "",
|
ca_cert_path: str = "",
|
||||||
ca_key_path: str = "",
|
ca_key_path: str = "",
|
||||||
ssrf_ip_allowlist: tuple[str, ...] = (),
|
ssrf_ip_allowlist: tuple[str, ...] = (),
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> dict[str, object]:
|
) -> dict[str, object]:
|
||||||
"""Build the structured pipelock config dict the sidecar will load.
|
"""Build the structured pipelock config dict the sidecar will load.
|
||||||
|
|
||||||
@@ -171,7 +178,7 @@ def pipelock_build_config(
|
|||||||
"version": 1,
|
"version": 1,
|
||||||
"mode": "strict",
|
"mode": "strict",
|
||||||
"enforce": True,
|
"enforce": True,
|
||||||
"api_allowlist": pipelock_effective_allowlist(bottle),
|
"api_allowlist": pipelock_effective_allowlist(bottle, provider_routes),
|
||||||
"forward_proxy": {"enabled": True},
|
"forward_proxy": {"enabled": True},
|
||||||
}
|
}
|
||||||
if not pipelock_seed_phrase_detection_enabled(bottle):
|
if not pipelock_seed_phrase_detection_enabled(bottle):
|
||||||
@@ -205,7 +212,7 @@ def pipelock_build_config(
|
|||||||
"enabled": True,
|
"enabled": True,
|
||||||
"ca_cert": ca_cert_path,
|
"ca_cert": ca_cert_path,
|
||||||
"ca_key": ca_key_path,
|
"ca_key": ca_key_path,
|
||||||
"passthrough_domains": pipelock_effective_tls_passthrough(bottle),
|
"passthrough_domains": pipelock_effective_tls_passthrough(bottle, provider_routes),
|
||||||
}
|
}
|
||||||
effective_ssrf_ip_allowlist = pipelock_effective_ssrf_ip_allowlist(
|
effective_ssrf_ip_allowlist = pipelock_effective_ssrf_ip_allowlist(
|
||||||
bottle, ssrf_ip_allowlist,
|
bottle, ssrf_ip_allowlist,
|
||||||
@@ -215,6 +222,180 @@ def pipelock_build_config(
|
|||||||
return cfg
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
_PIPELOCK_TOP_LEVEL_KEYS = {
|
||||||
|
"version",
|
||||||
|
"mode",
|
||||||
|
"enforce",
|
||||||
|
"api_allowlist",
|
||||||
|
"seed_phrase_detection",
|
||||||
|
"forward_proxy",
|
||||||
|
"dlp",
|
||||||
|
"request_body_scanning",
|
||||||
|
"tls_interception",
|
||||||
|
"ssrf",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _pipelock_render_error(section: str, key: str, expected: str) -> ValueError:
|
||||||
|
return ValueError(
|
||||||
|
f"pipelock_render_yaml: {section}.{key} must be {expected}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_unknown_keys(
|
||||||
|
section: str,
|
||||||
|
obj: dict[str, object],
|
||||||
|
allowed: set[str],
|
||||||
|
) -> None:
|
||||||
|
for key in sorted(set(obj) - allowed):
|
||||||
|
raise ValueError(f"pipelock_render_yaml: {section}.{key} is unsupported")
|
||||||
|
|
||||||
|
|
||||||
|
def _required_dict(
|
||||||
|
obj: dict[str, object],
|
||||||
|
section: str,
|
||||||
|
key: str,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
value = obj.get(key)
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise _pipelock_render_error(section, key, "a mapping")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required_bool(obj: dict[str, object], section: str, key: str) -> bool:
|
||||||
|
value = obj.get(key)
|
||||||
|
if not isinstance(value, bool):
|
||||||
|
raise _pipelock_render_error(section, key, "a boolean")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required_int(obj: dict[str, object], section: str, key: str) -> int:
|
||||||
|
value = obj.get(key)
|
||||||
|
if isinstance(value, bool) or not isinstance(value, int):
|
||||||
|
raise _pipelock_render_error(section, key, "an integer")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required_str(obj: dict[str, object], section: str, key: str) -> str:
|
||||||
|
value = obj.get(key)
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise _pipelock_render_error(section, key, "a string")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required_str_list(
|
||||||
|
obj: dict[str, object],
|
||||||
|
section: str,
|
||||||
|
key: str,
|
||||||
|
) -> list[str]:
|
||||||
|
value = obj.get(key)
|
||||||
|
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
|
||||||
|
raise _pipelock_render_error(section, key, "a list of strings")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_str_list(
|
||||||
|
obj: dict[str, object],
|
||||||
|
section: str,
|
||||||
|
key: str,
|
||||||
|
) -> list[str]:
|
||||||
|
if key not in obj:
|
||||||
|
return []
|
||||||
|
return _required_str_list(obj, section, key)
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_bool(
|
||||||
|
obj: dict[str, object],
|
||||||
|
section: str,
|
||||||
|
key: str,
|
||||||
|
) -> bool | None:
|
||||||
|
if key not in obj:
|
||||||
|
return None
|
||||||
|
return _required_bool(obj, section, key)
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_str(
|
||||||
|
obj: dict[str, object],
|
||||||
|
section: str,
|
||||||
|
key: str,
|
||||||
|
) -> str | None:
|
||||||
|
if key not in obj:
|
||||||
|
return None
|
||||||
|
return _required_str(obj, section, key)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_pipelock_render_config(cfg: dict[str, object]) -> dict[str, object]:
|
||||||
|
_reject_unknown_keys("config", cfg, _PIPELOCK_TOP_LEVEL_KEYS)
|
||||||
|
normalized: dict[str, object] = {
|
||||||
|
"version": _required_int(cfg, "config", "version"),
|
||||||
|
"mode": _required_str(cfg, "config", "mode"),
|
||||||
|
"enforce": _required_bool(cfg, "config", "enforce"),
|
||||||
|
"api_allowlist": _required_str_list(cfg, "config", "api_allowlist"),
|
||||||
|
}
|
||||||
|
|
||||||
|
if "seed_phrase_detection" in cfg:
|
||||||
|
spd = _required_dict(cfg, "config", "seed_phrase_detection")
|
||||||
|
_reject_unknown_keys("seed_phrase_detection", spd, {"enabled"})
|
||||||
|
normalized["seed_phrase_detection"] = {
|
||||||
|
"enabled": _required_bool(spd, "seed_phrase_detection", "enabled"),
|
||||||
|
}
|
||||||
|
|
||||||
|
fp = _required_dict(cfg, "config", "forward_proxy")
|
||||||
|
_reject_unknown_keys("forward_proxy", fp, {"enabled"})
|
||||||
|
normalized["forward_proxy"] = {
|
||||||
|
"enabled": _required_bool(fp, "forward_proxy", "enabled"),
|
||||||
|
}
|
||||||
|
|
||||||
|
dlp = _required_dict(cfg, "config", "dlp")
|
||||||
|
_reject_unknown_keys("dlp", dlp, {"include_defaults", "scan_env"})
|
||||||
|
normalized["dlp"] = {
|
||||||
|
"include_defaults": _required_bool(dlp, "dlp", "include_defaults"),
|
||||||
|
"scan_env": _required_bool(dlp, "dlp", "scan_env"),
|
||||||
|
}
|
||||||
|
|
||||||
|
rbs = _required_dict(cfg, "config", "request_body_scanning")
|
||||||
|
_reject_unknown_keys(
|
||||||
|
"request_body_scanning",
|
||||||
|
rbs,
|
||||||
|
{"action", "scan_headers", "header_mode"},
|
||||||
|
)
|
||||||
|
normalized_rbs: dict[str, object] = {
|
||||||
|
"action": _required_str(rbs, "request_body_scanning", "action"),
|
||||||
|
}
|
||||||
|
scan_headers = _optional_bool(rbs, "request_body_scanning", "scan_headers")
|
||||||
|
if scan_headers is not None:
|
||||||
|
normalized_rbs["scan_headers"] = scan_headers
|
||||||
|
header_mode = _optional_str(rbs, "request_body_scanning", "header_mode")
|
||||||
|
if header_mode is not None:
|
||||||
|
normalized_rbs["header_mode"] = header_mode
|
||||||
|
normalized["request_body_scanning"] = normalized_rbs
|
||||||
|
|
||||||
|
if "tls_interception" in cfg:
|
||||||
|
tls = _required_dict(cfg, "config", "tls_interception")
|
||||||
|
_reject_unknown_keys(
|
||||||
|
"tls_interception",
|
||||||
|
tls,
|
||||||
|
{"enabled", "ca_cert", "ca_key", "passthrough_domains"},
|
||||||
|
)
|
||||||
|
normalized["tls_interception"] = {
|
||||||
|
"enabled": _required_bool(tls, "tls_interception", "enabled"),
|
||||||
|
"ca_cert": _required_str(tls, "tls_interception", "ca_cert"),
|
||||||
|
"ca_key": _required_str(tls, "tls_interception", "ca_key"),
|
||||||
|
"passthrough_domains": _optional_str_list(
|
||||||
|
tls, "tls_interception", "passthrough_domains",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
if "ssrf" in cfg:
|
||||||
|
ssrf = _required_dict(cfg, "config", "ssrf")
|
||||||
|
_reject_unknown_keys("ssrf", ssrf, {"ip_allowlist"})
|
||||||
|
normalized["ssrf"] = {
|
||||||
|
"ip_allowlist": _required_str_list(ssrf, "ssrf", "ip_allowlist"),
|
||||||
|
}
|
||||||
|
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||||
"""Render a pipelock config dict (as produced by
|
"""Render a pipelock config dict (as produced by
|
||||||
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
||||||
@@ -222,31 +403,38 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
def _bool(b: object) -> str:
|
def _bool(b: object) -> str:
|
||||||
return "true" if b else "false"
|
return "true" if b else "false"
|
||||||
|
|
||||||
|
cfg = _validate_pipelock_render_config(cfg)
|
||||||
lines: list[str] = []
|
lines: list[str] = []
|
||||||
lines.append(f"version: {cfg['version']}")
|
lines.append(f"version: {cfg['version']}")
|
||||||
lines.append(f"mode: {cfg['mode']}")
|
lines.append(f"mode: {cfg['mode']}")
|
||||||
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("api_allowlist:")
|
lines.append("api_allowlist:")
|
||||||
for h in cast(list[str], cfg["api_allowlist"]):
|
api_allowlist = cfg["api_allowlist"]
|
||||||
|
assert isinstance(api_allowlist, list)
|
||||||
|
for h in api_allowlist:
|
||||||
lines.append(f' - "{h}"')
|
lines.append(f' - "{h}"')
|
||||||
lines.append("")
|
lines.append("")
|
||||||
if "seed_phrase_detection" in cfg:
|
if "seed_phrase_detection" in cfg:
|
||||||
lines.append("seed_phrase_detection:")
|
lines.append("seed_phrase_detection:")
|
||||||
spd = cast(dict[str, object], cfg["seed_phrase_detection"])
|
spd = cfg["seed_phrase_detection"]
|
||||||
|
assert isinstance(spd, dict)
|
||||||
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("forward_proxy:")
|
lines.append("forward_proxy:")
|
||||||
fp = cast(dict[str, object], cfg["forward_proxy"])
|
fp = cfg["forward_proxy"]
|
||||||
|
assert isinstance(fp, dict)
|
||||||
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("dlp:")
|
lines.append("dlp:")
|
||||||
dlp = cast(dict[str, object], cfg["dlp"])
|
dlp = cfg["dlp"]
|
||||||
|
assert isinstance(dlp, dict)
|
||||||
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
||||||
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("request_body_scanning:")
|
lines.append("request_body_scanning:")
|
||||||
rbs = cast(dict[str, object], cfg["request_body_scanning"])
|
rbs = cfg["request_body_scanning"]
|
||||||
|
assert isinstance(rbs, dict)
|
||||||
lines.append(f' action: "{rbs["action"]}"')
|
lines.append(f' action: "{rbs["action"]}"')
|
||||||
if "scan_headers" in rbs:
|
if "scan_headers" in rbs:
|
||||||
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
||||||
@@ -255,11 +443,13 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
if "tls_interception" in cfg:
|
if "tls_interception" in cfg:
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("tls_interception:")
|
lines.append("tls_interception:")
|
||||||
tls = cast(dict[str, object], cfg["tls_interception"])
|
tls = cfg["tls_interception"]
|
||||||
|
assert isinstance(tls, dict)
|
||||||
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
||||||
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
||||||
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
||||||
passthrough = cast(list[str], tls.get("passthrough_domains", []))
|
passthrough = tls["passthrough_domains"]
|
||||||
|
assert isinstance(passthrough, list)
|
||||||
if passthrough:
|
if passthrough:
|
||||||
lines.append(" passthrough_domains:")
|
lines.append(" passthrough_domains:")
|
||||||
for d in passthrough:
|
for d in passthrough:
|
||||||
@@ -267,9 +457,12 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
if "ssrf" in cfg:
|
if "ssrf" in cfg:
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("ssrf:")
|
lines.append("ssrf:")
|
||||||
ssrf = cast(dict[str, object], cfg["ssrf"])
|
ssrf = cfg["ssrf"]
|
||||||
|
assert isinstance(ssrf, dict)
|
||||||
lines.append(" ip_allowlist:")
|
lines.append(" ip_allowlist:")
|
||||||
for ip in cast(list[str], ssrf["ip_allowlist"]):
|
ip_allowlist = ssrf["ip_allowlist"]
|
||||||
|
assert isinstance(ip_allowlist, list)
|
||||||
|
for ip in ip_allowlist:
|
||||||
lines.append(f' - "{ip}"')
|
lines.append(f' - "{ip}"')
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
@@ -319,7 +512,11 @@ class PipelockProxy:
|
|||||||
(`PIPELOCK_CA_CERT_IN_CONTAINER` / `PIPELOCK_CA_KEY_IN_CONTAINER`)."""
|
(`PIPELOCK_CA_CERT_IN_CONTAINER` / `PIPELOCK_CA_KEY_IN_CONTAINER`)."""
|
||||||
|
|
||||||
def prepare(
|
def prepare(
|
||||||
self, bottle: Bottle, slug: str, stage_dir: Path
|
self,
|
||||||
|
bottle: Bottle,
|
||||||
|
slug: str,
|
||||||
|
stage_dir: Path,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> PipelockProxyPlan:
|
) -> PipelockProxyPlan:
|
||||||
"""Write the pipelock yaml config (mode 600) under `stage_dir`
|
"""Write the pipelock yaml config (mode 600) under `stage_dir`
|
||||||
and return the plan for launch. Pure host-side, no docker
|
and return the plan for launch. Pure host-side, no docker
|
||||||
@@ -342,6 +539,7 @@ class PipelockProxy:
|
|||||||
bottle,
|
bottle,
|
||||||
ca_cert_path=PIPELOCK_CA_CERT_IN_CONTAINER,
|
ca_cert_path=PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||||
ca_key_path=PIPELOCK_CA_KEY_IN_CONTAINER,
|
ca_key_path=PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||||
|
provider_routes=provider_routes,
|
||||||
)
|
)
|
||||||
yaml_path.write_text(pipelock_render_yaml(cfg))
|
yaml_path.write_text(pipelock_render_yaml(cfg))
|
||||||
yaml_path.chmod(0o600)
|
yaml_path.chmod(0o600)
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ sick daemon."
|
|||||||
|
|
||||||
Daemon subset is env-driven. The compose renderer narrows it via
|
Daemon subset is env-driven. The compose renderer narrows it via
|
||||||
`BOT_BOTTLE_SIDECAR_DAEMONS=egress,pipelock` for bottles that
|
`BOT_BOTTLE_SIDECAR_DAEMONS=egress,pipelock` for bottles that
|
||||||
don't use git-gate or supervise. Default: all four.
|
don't use git-gate or supervise. Default: all daemons.
|
||||||
|
|
||||||
Stdlib-only by design — adding supervisord/s6/runit for four
|
Stdlib-only by design — adding supervisord/s6/runit for four
|
||||||
daemons is heavier than this script.
|
daemons is heavier than this script.
|
||||||
@@ -98,6 +98,7 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
|
|||||||
"--listen", "0.0.0.0:8888"),
|
"--listen", "0.0.0.0:8888"),
|
||||||
),
|
),
|
||||||
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
|
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
|
||||||
|
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")),
|
||||||
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
|
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -162,6 +163,10 @@ class _Supervisor:
|
|||||||
# Names of children that have been logged as having exited
|
# Names of children that have been logged as having exited
|
||||||
# so we only log each death once across watch-loop ticks.
|
# so we only log each death once across watch-loop ticks.
|
||||||
self._logged_dead: set[str] = set()
|
self._logged_dead: set[str] = set()
|
||||||
|
# Signal handlers add daemon names here and return quickly.
|
||||||
|
# The main watch loop drains the set, so repeated restart
|
||||||
|
# requests for one daemon coalesce into one restart.
|
||||||
|
self._restart_requested: set[str] = set()
|
||||||
|
|
||||||
def start_all(self) -> None:
|
def start_all(self) -> None:
|
||||||
for spec in self.specs:
|
for spec in self.specs:
|
||||||
@@ -172,6 +177,7 @@ class _Supervisor:
|
|||||||
if self.shutdown_at is not None:
|
if self.shutdown_at is not None:
|
||||||
return
|
return
|
||||||
self.shutdown_at = time.monotonic()
|
self.shutdown_at = time.monotonic()
|
||||||
|
self._restart_requested.clear()
|
||||||
_log(f"shutting down ({reason}); forwarding SIGTERM")
|
_log(f"shutting down ({reason}); forwarding SIGTERM")
|
||||||
for _, p in self.procs:
|
for _, p in self.procs:
|
||||||
if p.poll() is None:
|
if p.poll() is None:
|
||||||
@@ -180,6 +186,24 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
def request_restart(self, daemon_name: str) -> bool:
|
||||||
|
"""Queue a daemon restart for the main loop to process.
|
||||||
|
|
||||||
|
Signal handlers use this non-blocking path instead of doing
|
||||||
|
subprocess lifecycle work directly. Requests coalesce by
|
||||||
|
daemon name: one pending restart is enough to make the daemon
|
||||||
|
reread the latest config from disk.
|
||||||
|
|
||||||
|
Returns True iff a daemon by that name is known to the
|
||||||
|
supervisor and shutdown has not started."""
|
||||||
|
if self.shutdown_at is not None:
|
||||||
|
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
|
||||||
|
return False
|
||||||
|
if not any(spec.name == daemon_name for spec, _ in self.procs):
|
||||||
|
return False
|
||||||
|
self._restart_requested.add(daemon_name)
|
||||||
|
return True
|
||||||
|
|
||||||
def tick(self) -> bool:
|
def tick(self) -> bool:
|
||||||
"""One iteration of the watch loop. Returns True when every
|
"""One iteration of the watch loop. Returns True when every
|
||||||
child has exited and the supervisor can return.
|
child has exited and the supervisor can return.
|
||||||
@@ -187,6 +211,8 @@ class _Supervisor:
|
|||||||
A child dying unexpectedly is logged but does NOT initiate
|
A child dying unexpectedly is logged but does NOT initiate
|
||||||
shutdown — see the module docstring's failure-policy
|
shutdown — see the module docstring's failure-policy
|
||||||
section. Shutdown is signal-driven only."""
|
section. Shutdown is signal-driven only."""
|
||||||
|
self._drain_restart_requests()
|
||||||
|
|
||||||
for spec, p in self.procs:
|
for spec, p in self.procs:
|
||||||
rc = p.poll()
|
rc = p.poll()
|
||||||
if rc is None or spec.name in self._logged_dead:
|
if rc is None or spec.name in self._logged_dead:
|
||||||
@@ -219,14 +245,37 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
return all(p.poll() is not None for _, p in self.procs)
|
done = all(p.poll() is not None for _, p in self.procs)
|
||||||
|
if done:
|
||||||
|
for _, p in self.procs:
|
||||||
|
if p.stdout is not None:
|
||||||
|
p.stdout.close()
|
||||||
|
return done
|
||||||
|
|
||||||
def exit_code(self) -> int:
|
def exit_code(self) -> int:
|
||||||
"""Worst child returncode wins. On graceful shutdown every
|
"""Positive child failures win; otherwise report success.
|
||||||
child is signal-killed (negative returncode) and max()
|
|
||||||
returns 0; if some child crashed nonzero before the signal
|
Python represents signal-terminated children as negative
|
||||||
the operator gets that code on container exit."""
|
return codes. A signal-only graceful shutdown should not leak
|
||||||
return max((p.returncode for _, p in self.procs), default=0)
|
that platform-specific detail into the container exit status,
|
||||||
|
but a positive crash before shutdown should remain visible."""
|
||||||
|
positives = [
|
||||||
|
p.returncode for _, p in self.procs
|
||||||
|
if p.returncode is not None and p.returncode > 0
|
||||||
|
]
|
||||||
|
return max(positives, default=0)
|
||||||
|
|
||||||
|
def _drain_restart_requests(self) -> None:
|
||||||
|
if self.shutdown_at is not None:
|
||||||
|
self._restart_requested.clear()
|
||||||
|
return
|
||||||
|
requested = tuple(sorted(self._restart_requested))
|
||||||
|
self._restart_requested.clear()
|
||||||
|
for daemon_name in requested:
|
||||||
|
if self.shutdown_at is not None:
|
||||||
|
self._restart_requested.clear()
|
||||||
|
return
|
||||||
|
self.restart_daemon(daemon_name)
|
||||||
|
|
||||||
def forward_signal(self, sig: int, daemon_name: str) -> bool:
|
def forward_signal(self, sig: int, daemon_name: str) -> bool:
|
||||||
"""Forward a signal to one named child. Used by the SIGHUP
|
"""Forward a signal to one named child. Used by the SIGHUP
|
||||||
@@ -291,6 +340,8 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
p.wait()
|
p.wait()
|
||||||
|
if p.stdout is not None:
|
||||||
|
p.stdout.close()
|
||||||
self._logged_dead.discard(daemon_name)
|
self._logged_dead.discard(daemon_name)
|
||||||
new_proc = _spawn(spec)
|
new_proc = _spawn(spec)
|
||||||
self.procs[idx] = (spec, new_proc)
|
self.procs[idx] = (spec, new_proc)
|
||||||
@@ -322,7 +373,7 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|||||||
# supervisor restarts the pipelock daemon in place (other
|
# supervisor restarts the pipelock daemon in place (other
|
||||||
# daemons keep running — specifically supervise, whose MCP
|
# daemons keep running — specifically supervise, whose MCP
|
||||||
# socket would drop on a whole-container `docker restart`).
|
# socket would drop on a whole-container `docker restart`).
|
||||||
signal.signal(signal.SIGUSR1, lambda *_: sup.restart_daemon("pipelock"))
|
signal.signal(signal.SIGUSR1, lambda *_: sup.request_restart("pipelock"))
|
||||||
|
|
||||||
while not sup.tick():
|
while not sup.tick():
|
||||||
time.sleep(_POLL_INTERVAL)
|
time.sleep(_POLL_INTERVAL)
|
||||||
|
|||||||
@@ -35,6 +35,7 @@ import json
|
|||||||
import os
|
import os
|
||||||
import socketserver
|
import socketserver
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
import typing
|
import typing
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
@@ -63,6 +64,10 @@ ERR_METHOD_NOT_FOUND = -32601
|
|||||||
ERR_INVALID_PARAMS = -32602
|
ERR_INVALID_PARAMS = -32602
|
||||||
ERR_INTERNAL = -32603
|
ERR_INTERNAL = -32603
|
||||||
|
|
||||||
|
DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
|
||||||
|
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
|
||||||
|
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class JsonRpcRequest:
|
class JsonRpcRequest:
|
||||||
@@ -412,6 +417,7 @@ def _validate_and_bundle_egress_route(
|
|||||||
class ServerConfig:
|
class ServerConfig:
|
||||||
bottle_slug: str
|
bottle_slug: str
|
||||||
queue_dir: Path
|
queue_dir: Path
|
||||||
|
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
||||||
|
|
||||||
|
|
||||||
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
|
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
|
||||||
@@ -442,7 +448,7 @@ def handle_list_egress_routes(
|
|||||||
})
|
})
|
||||||
opener = urllib.request.build_opener(proxy_handler)
|
opener = urllib.request.build_opener(proxy_handler)
|
||||||
try:
|
try:
|
||||||
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=5) as resp:
|
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=EGRESS_LIST_TIMEOUT_SECONDS) as resp:
|
||||||
body = resp.read().decode("utf-8")
|
body = resp.read().decode("utf-8")
|
||||||
except (urllib.error.URLError, OSError) as e:
|
except (urllib.error.URLError, OSError) as e:
|
||||||
return {
|
return {
|
||||||
@@ -520,7 +526,20 @@ def handle_tools_call(
|
|||||||
f"for bottle {config.bottle_slug}; waiting for operator...\n"
|
f"for bottle {config.bottle_slug}; waiting for operator...\n"
|
||||||
)
|
)
|
||||||
sys.stderr.flush()
|
sys.stderr.flush()
|
||||||
response = _sv.wait_for_response(config.queue_dir, proposal.id)
|
deadline = time.monotonic() + config.response_timeout_seconds
|
||||||
|
try:
|
||||||
|
response = _sv.wait_for_response(
|
||||||
|
config.queue_dir,
|
||||||
|
proposal.id,
|
||||||
|
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
|
||||||
|
deadline=deadline,
|
||||||
|
)
|
||||||
|
except TimeoutError:
|
||||||
|
text = format_pending_response_text(config.response_timeout_seconds)
|
||||||
|
return {
|
||||||
|
"content": [{"type": "text", "text": text}],
|
||||||
|
"isError": False,
|
||||||
|
}
|
||||||
_sv.archive_proposal(config.queue_dir, proposal.id)
|
_sv.archive_proposal(config.queue_dir, proposal.id)
|
||||||
|
|
||||||
text = format_response_text(response)
|
text = format_response_text(response)
|
||||||
@@ -542,6 +561,16 @@ def format_response_text(response: "_sv.Response") -> str:
|
|||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def format_pending_response_text(timeout_seconds: float) -> str:
|
||||||
|
return "\n".join([
|
||||||
|
"status: pending",
|
||||||
|
(
|
||||||
|
"notes: operator response timed out after "
|
||||||
|
f"{timeout_seconds:g}s; proposal remains queued"
|
||||||
|
),
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
# --- HTTP transport --------------------------------------------------------
|
# --- HTTP transport --------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@@ -654,10 +683,15 @@ def serve(
|
|||||||
queue_dir: Path,
|
queue_dir: Path,
|
||||||
port: int = _sv.SUPERVISE_PORT,
|
port: int = _sv.SUPERVISE_PORT,
|
||||||
bind: str = "0.0.0.0",
|
bind: str = "0.0.0.0",
|
||||||
|
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
|
||||||
) -> typing.NoReturn:
|
) -> typing.NoReturn:
|
||||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||||
server = MCPServer((bind, port), MCPHandler)
|
server = MCPServer((bind, port), MCPHandler)
|
||||||
server.config = ServerConfig(bottle_slug=bottle_slug, queue_dir=queue_dir)
|
server.config = ServerConfig(
|
||||||
|
bottle_slug=bottle_slug,
|
||||||
|
queue_dir=queue_dir,
|
||||||
|
response_timeout_seconds=response_timeout_seconds,
|
||||||
|
)
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
f"supervise listening on {bind}:{port}; "
|
f"supervise listening on {bind}:{port}; "
|
||||||
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
||||||
@@ -682,9 +716,37 @@ def main(argv: list[str]) -> int:
|
|||||||
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
||||||
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
||||||
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
||||||
serve(bottle_slug=bottle_slug, queue_dir=queue_dir, port=port, bind=bind)
|
try:
|
||||||
|
response_timeout_seconds = _response_timeout_from_env(os.environ)
|
||||||
|
except ValueError as e:
|
||||||
|
sys.stderr.write(f"supervise: {e}\n")
|
||||||
|
return 2
|
||||||
|
serve(
|
||||||
|
bottle_slug=bottle_slug,
|
||||||
|
queue_dir=queue_dir,
|
||||||
|
port=port,
|
||||||
|
bind=bind,
|
||||||
|
response_timeout_seconds=response_timeout_seconds,
|
||||||
|
)
|
||||||
return 0 # serve() does not return
|
return 0 # serve() does not return
|
||||||
|
|
||||||
|
|
||||||
|
def _response_timeout_from_env(env: typing.Mapping[str, str]) -> float:
|
||||||
|
raw = env.get("SUPERVISE_RESPONSE_TIMEOUT_SECONDS", "").strip()
|
||||||
|
if not raw:
|
||||||
|
return DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
||||||
|
try:
|
||||||
|
value = float(raw)
|
||||||
|
except ValueError as e:
|
||||||
|
raise ValueError(
|
||||||
|
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
||||||
|
) from e
|
||||||
|
if value <= 0:
|
||||||
|
raise ValueError(
|
||||||
|
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
||||||
|
)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
raise SystemExit(main(sys.argv))
|
raise SystemExit(main(sys.argv))
|
||||||
|
|||||||
@@ -5,9 +5,18 @@ level deeper, under their backend package."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ipaddress
|
||||||
import os
|
import os
|
||||||
|
|
||||||
|
|
||||||
|
def is_ip_literal(value: str) -> bool:
|
||||||
|
try:
|
||||||
|
ipaddress.ip_address(value)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
def expand_tilde(path: str) -> str:
|
def expand_tilde(path: str) -> str:
|
||||||
"""Expand a leading '~' to $HOME. Leaves paths without a leading
|
"""Expand a leading '~' to $HOME. Leaves paths without a leading
|
||||||
tilde unchanged. Falls back to the empty string if $HOME is unset
|
tilde unchanged. Falls back to the empty string if $HOME is unset
|
||||||
|
|||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""Backend-neutral plan for porting the operator workspace."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Protocol
|
||||||
|
|
||||||
|
|
||||||
|
WORKSPACE_DIRNAME = "workspace"
|
||||||
|
DEFAULT_WORKSPACE_OWNER = "node:node"
|
||||||
|
DEFAULT_WORKSPACE_MODE = "755"
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceSpec(Protocol):
|
||||||
|
copy_cwd: bool
|
||||||
|
user_cwd: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WorkspacePlan:
|
||||||
|
"""Resolved workspace contract shared by all bottle backends."""
|
||||||
|
|
||||||
|
enabled: bool
|
||||||
|
host_path: Path
|
||||||
|
guest_home: str
|
||||||
|
guest_path: str
|
||||||
|
workdir: str
|
||||||
|
owner: str = DEFAULT_WORKSPACE_OWNER
|
||||||
|
mode: str = DEFAULT_WORKSPACE_MODE
|
||||||
|
copy_contents: bool = True
|
||||||
|
copy_git: bool = True
|
||||||
|
has_host_git_dir: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
def workspace_plan(spec: WorkspaceSpec, *, guest_home: str) -> WorkspacePlan:
|
||||||
|
"""Resolve the in-bottle workspace path from CLI intent."""
|
||||||
|
host_path = Path(spec.user_cwd).expanduser()
|
||||||
|
if spec.copy_cwd:
|
||||||
|
guest_path = f"{guest_home.rstrip('/')}/{WORKSPACE_DIRNAME}"
|
||||||
|
workdir = guest_path
|
||||||
|
else:
|
||||||
|
guest_path = guest_home
|
||||||
|
workdir = guest_home
|
||||||
|
return WorkspacePlan(
|
||||||
|
enabled=spec.copy_cwd,
|
||||||
|
host_path=host_path,
|
||||||
|
guest_home=guest_home,
|
||||||
|
guest_path=guest_path,
|
||||||
|
workdir=workdir,
|
||||||
|
has_host_git_dir=(host_path / ".git").is_dir(),
|
||||||
|
)
|
||||||
@@ -1 +0,0 @@
|
|||||||
Research notes live in `research/`. Product requirement docs live in `prds/`.
|
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# Docs
|
||||||
|
|
||||||
|
How this project records what it builds and why — and a guide to
|
||||||
|
picking the right document for what you're capturing.
|
||||||
|
|
||||||
|
## When to write which document
|
||||||
|
|
||||||
|
| Artifact | For |
|
||||||
|
|---|---|
|
||||||
|
| **PRD** (`docs/prds/`) | A feature: what to build, scope, success criteria. |
|
||||||
|
| **Research note** (`docs/research/`) | A landscape/tradeoff investigation. |
|
||||||
|
| **Decision record** (`docs/decisions/`) | A decision that isn't itself a feature — a policy, a convention, a "we will / won't do this," or a load-bearing choice made inside a larger PRD that deserves to be discoverable on its own. |
|
||||||
|
|
||||||
|
A decision that's fully specified by a PRD doesn't need duplicating in
|
||||||
|
a decision record. Write one when the *decision* would otherwise be
|
||||||
|
buried in prose, lost in an issue thread, or have no in-repo home at
|
||||||
|
all (small requests that don't merit a PRD; non-feature choices like
|
||||||
|
merge strategy or a trust posture).
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# ADR 0001: Merge PRs with rebase, not merge commits
|
||||||
|
|
||||||
|
- **Status:** Accepted
|
||||||
|
- **Date:** 2026-05-28
|
||||||
|
- **Deciders:** didericis
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
PRs need a merge strategy. Gitea offers merge-commit, squash, rebase,
|
||||||
|
and rebase-merge. The project uses [Conventional
|
||||||
|
Commits](https://www.conventionalcommits.org/) enforced by a
|
||||||
|
`commit-msg` hook, and PRDs typically land as a multi-commit PR where
|
||||||
|
each commit is meaningful on its own (e.g. PR #95: a `docs(prd)` commit,
|
||||||
|
a `feat(manifest)` implementation commit, and a `docs(manifest)`
|
||||||
|
commit). The history should stay readable and the individual
|
||||||
|
conventional commits should survive onto `main`.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Merge PRs with **rebase** (Gitea's `rebase` style; `Do: "rebase"` via
|
||||||
|
the API). The branch's commits are replayed onto `main` with no merge
|
||||||
|
commit, producing a linear history that preserves each commit verbatim.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- **Linear history**, no merge bubbles; `git log --oneline` reads as a
|
||||||
|
straight sequence of conventional commits.
|
||||||
|
- **Each commit is preserved** (unlike squash, which would collapse the
|
||||||
|
PRD/impl/docs commits into one and lose the staged structure).
|
||||||
|
- **Commit SHAs are rewritten at merge.** The replayed commits on `main`
|
||||||
|
get new SHAs, and the source branch is deleted, so a link to a file
|
||||||
|
by *branch name* (`/src/branch/<feature>/…`) dies at merge. This is
|
||||||
|
why links to not-yet-merged files are pinned to a **commit SHA**
|
||||||
|
(`/src/commit/<sha>/…`), which stays reachable via the retained
|
||||||
|
`refs/pull/<n>/head` ref. See
|
||||||
|
`docs/research/issue-tracking-vs-in-repo-decision-history.md`.
|
||||||
|
- **Trade-off accepted:** without a merge commit, the "these commits
|
||||||
|
landed together as PR #N" grouping is not recorded in git itself — it
|
||||||
|
lives in forge state (the PR). That is a mild concession against the
|
||||||
|
keep-history-in-the-repo posture; the conventional-commit scopes and
|
||||||
|
PRD references in the messages keep changes traceable without it.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- `docs/research/issue-tracking-vs-in-repo-decision-history.md` — the
|
||||||
|
commit-pinning consequence above.
|
||||||
|
- Observed practice: PRs #92, #93 merged with rebase; #95 to follow.
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
# ADR 0002: Agent-set git identity is claimed, not vouched
|
||||||
|
|
||||||
|
- **Status:** Accepted
|
||||||
|
- **Date:** 2026-05-28
|
||||||
|
- **Deciders:** didericis
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
PRD 0027 lifts `git.user` (name/email) to the agent layer, so an agent
|
||||||
|
file may declare its own commit identity. Agent files can live in
|
||||||
|
`$CWD/.bot-bottle/agents/` — i.e. they can be supplied by a cloned,
|
||||||
|
less-trusted repository. That raises the question of whether a
|
||||||
|
repo-supplied agent setting its own git identity is a security concern,
|
||||||
|
and whether agent identity should be gated differently for `$CWD`
|
||||||
|
agents than for `$HOME` agents.
|
||||||
|
|
||||||
|
This record exists because the decision is a **trust posture** worth
|
||||||
|
finding on its own, separate from the feature PRD that introduced it.
|
||||||
|
The full analysis lives in PRD 0027; the decision is summarized here.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Allow agents to set `git.user`, and treat an agent-declared identity as
|
||||||
|
**claimed, not vouched**. No `$CWD`-vs-`$HOME` gating on the identity
|
||||||
|
field. `git.remotes` stays bottle-only (home-only).
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- A cloned repo's agent file can present any commit author name/email,
|
||||||
|
including one that reads like a real person's. This is accepted: git
|
||||||
|
authorship is **not a credential** (push auth is the bottle's remote
|
||||||
|
key/token), is **already forgeable** from inside the bottle at runtime
|
||||||
|
(`git config user.email …`), and was never a trust anchor.
|
||||||
|
- If attribution integrity ever matters, the answer is commit
|
||||||
|
**signing** (SSH/GPG), not the author field — so this decision closes
|
||||||
|
no door that was open.
|
||||||
|
- `git.remotes` is deliberately *not* lifted to the agent layer: it
|
||||||
|
carries credentials and host trust (IdentityFile, KnownHostKey) and
|
||||||
|
remains a bottle-only, home-only concern.
|
||||||
|
- Revisit if a future change ever makes commit identity load-bearing
|
||||||
|
(e.g. enforced signing keyed on author), at which point gating
|
||||||
|
`$CWD`-supplied identities would matter.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- PRD 0027 (`docs/prds/0027-agent-git-user-identity.md`) — full trust
|
||||||
|
analysis and schema.
|
||||||
|
- Issue #94, PR #95 — the feature this decision was made for.
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# ADR 0003: Keep agent system prompts user-directed, not auto-generated from config
|
||||||
|
|
||||||
|
- **Status:** Accepted
|
||||||
|
- **Date:** 2026-05-29
|
||||||
|
- **Deciders:** didericis
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
A bottle already declares exactly what an agent can reach: egress routes
|
||||||
|
(allowlisted hosts + auth) and git config (remotes + identity). We
|
||||||
|
considered deriving an agent's system prompt — or a section of it —
|
||||||
|
automatically from those configs, so an agent would be told up front
|
||||||
|
what it has access to (e.g. "you can reach `gitea.dideric.is` over the
|
||||||
|
git remote and its API"). The question surfaced while hand-writing that
|
||||||
|
exact line into the `claude-implementer` prompt.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
System prompts stay **user-directed** — authored by the operator. We do
|
||||||
|
not auto-generate prompt content from a bottle's egress / git config.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The operator controls what the agent is *told* about its environment,
|
||||||
|
independently of what the bottle *grants*. Sometimes we may want to
|
||||||
|
withhold that information from the agent directly — keep the prompt
|
||||||
|
silent about an allowlisted host even though egress permits it.
|
||||||
|
- The agent can still infer its access on its own (attempt a request,
|
||||||
|
read its env, `git remote -v`, the gitconfig), so auto-injection is a
|
||||||
|
convenience, not a capability the agent depends on.
|
||||||
|
- Cost accepted: operators must restate access in the prompt when they
|
||||||
|
want the agent to know it (as we did for the Gitea instance), and the
|
||||||
|
prompt can drift from the config. That decoupling of "what the bottle
|
||||||
|
grants" from "what the agent is told" is the point.
|
||||||
|
- Revisit if keeping prompts in sync with configs becomes a real pain.
|
||||||
|
An *opt-in* helper that emits a capability summary the operator
|
||||||
|
chooses to include would honor this decision; silent auto-injection
|
||||||
|
would not.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- ADR 0002 (`0002-agent-identity-claimed-not-vouched.md`) — related
|
||||||
|
agent-trust posture (what the agent is granted vs. what it can claim).
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Decision records
|
||||||
|
|
||||||
|
Short, durable records of decisions — one file per decision. This is a
|
||||||
|
lightweight [Architecture Decision Record](https://adr.github.io/)
|
||||||
|
practice: capture *what was decided and why* in a versioned file so the
|
||||||
|
reasoning lives in the clone, not in a Gitea issue thread or a chat log
|
||||||
|
that disappears when the host does.
|
||||||
|
|
||||||
|
See `docs/research/issue-tracking-vs-in-repo-decision-history.md` for
|
||||||
|
the rationale behind keeping decision history in-repo, and
|
||||||
|
[`docs/README.md`](../README.md) for when to write a decision record
|
||||||
|
vs. a PRD or research note.
|
||||||
|
|
||||||
|
## Format
|
||||||
|
|
||||||
|
One Markdown file per decision, numbered sequentially and zero-padded
|
||||||
|
(`0001-…`, `0002-…`), matching the PRD numbering style. Keep it short —
|
||||||
|
the discipline is writing it down, not the ceremony.
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# ADR 0000: <short imperative title>
|
||||||
|
|
||||||
|
- **Status:** Proposed | Accepted | Superseded by ADR NNNN
|
||||||
|
- **Date:** YYYY-MM-DD
|
||||||
|
- **Deciders:** <who>
|
||||||
|
|
||||||
|
## Context
|
||||||
|
What forced the decision; the constraints in play.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
What we decided, stated plainly.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
What follows — the good, and the costs/trade-offs accepted.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
PRDs, research notes, issues/PRs. Gitea links are convenience
|
||||||
|
pointers; the reasoning above must stand without them.
|
||||||
|
```
|
||||||
|
|
||||||
|
The records are the index: `ls docs/decisions/` or skim the titles.
|
||||||
|
No hand-maintained list to keep in sync.
|
||||||
@@ -83,12 +83,7 @@ for a declared upstream:
|
|||||||
- **Manifest field.** `bottle.git` — a list of git remotes the
|
- **Manifest field.** `bottle.git` — a list of git remotes the
|
||||||
bottle is allowed to talk to, each with the credential the gate
|
bottle is allowed to talk to, each with the credential the gate
|
||||||
uses to push upstream. The agent gets no parallel `bottle.ssh`
|
uses to push upstream. The agent gets no parallel `bottle.ssh`
|
||||||
entry for those upstreams. Each entry may also carry an
|
entry for those upstreams.
|
||||||
`ExtraHosts: { hostname: ip }` map, surfaced to the gate as
|
|
||||||
`--add-host` so the gate can resolve upstreams whose public DNS
|
|
||||||
doesn't point at the reachable IP (e.g. Tailscale-only hosts).
|
|
||||||
The agent-side `insteadOf` rewrite keys off the original hostname,
|
|
||||||
so the manifest's `Upstream` URL stays human-readable.
|
|
||||||
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
|
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
|
||||||
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
|
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
|
||||||
operation against the declared upstream (push, fetch, clone,
|
operation against the declared upstream (push, fetch, clone,
|
||||||
|
|||||||
@@ -88,8 +88,7 @@ the unused path.
|
|||||||
- **Pipelock interaction.** Drop the SSH-derived branch from
|
- **Pipelock interaction.** Drop the SSH-derived branch from
|
||||||
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
|
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
|
||||||
there is no per-upstream IP carve-out to render; git-gate
|
there is no per-upstream IP carve-out to render; git-gate
|
||||||
has its own egress network and pulls in upstream resolution
|
has its own egress network.
|
||||||
via `ExtraHosts` plus DNS.
|
|
||||||
- **Tests.** Delete the ssh-gate unit + integration suites,
|
- **Tests.** Delete the ssh-gate unit + integration suites,
|
||||||
the ssh fixtures in `tests/fixtures.py`, and the
|
the ssh fixtures in `tests/fixtures.py`, and the
|
||||||
shadow-route assertions in `test_manifest_git.py`. Adjust
|
shadow-route assertions in `test_manifest_git.py`. Adjust
|
||||||
|
|||||||
@@ -274,8 +274,6 @@ git:
|
|||||||
Name: bot-bottle
|
Name: bot-bottle
|
||||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
||||||
ExtraHosts:
|
|
||||||
gitea.dideric.is: 100.78.141.42
|
|
||||||
KnownHostKey: ssh-rsa AAAAB3...
|
KnownHostKey: ssh-rsa AAAAB3...
|
||||||
egress:
|
egress:
|
||||||
allowlist:
|
allowlist:
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ A real stuck agent recovers end-to-end in each of the three categories: a **cred
|
|||||||
|
|
||||||
- Live attach or in-place mutation of running containers. The whole design exists to avoid this.
|
- Live attach or in-place mutation of running containers. The whole design exists to avoid this.
|
||||||
- Agent-to-agent communication. Re-stated from the project's existing non-goals; the recovery flow is human→agent only.
|
- Agent-to-agent communication. Re-stated from the project's existing non-goals; the recovery flow is human→agent only.
|
||||||
- Auditing or forensic replay of agent runs. Git/forge history is the audit log; this PRD does not add a separate run log.
|
- Auditing or forensic replay of agent runs. Git/Gitea history is the audit log; this PRD does not add a separate run log.
|
||||||
- Reducing time-to-unstuck below some target. Faster than kill-and-restart is implicit, but no specific SLO is in scope.
|
- Reducing time-to-unstuck below some target. Faster than kill-and-restart is implicit, but no specific SLO is in scope.
|
||||||
|
|
||||||
## Stuck categories
|
## Stuck categories
|
||||||
|
|||||||
@@ -39,6 +39,41 @@ trust boundary*: only `$HOME` bottles can declare it, only `$HOME`
|
|||||||
bottles can be its target. Cloned repos still cannot author
|
bottles can be its target. Cloned repos still cannot author
|
||||||
bottle-equivalent config.
|
bottle-equivalent config.
|
||||||
|
|
||||||
|
## Alternatives considered
|
||||||
|
|
||||||
|
The question raised in issue #88 was *where composition should live*.
|
||||||
|
Three points in that design space, recorded here so the decision
|
||||||
|
stands on its own without the issue thread:
|
||||||
|
|
||||||
|
1. **Duplicate bottles (status quo).** Copy `dev.md` to `staging.md`
|
||||||
|
and edit. Zero new mechanism, but every shared field drifts: a
|
||||||
|
route added to `dev` is silently missing from `staging`. This is
|
||||||
|
the pain that prompted #88.
|
||||||
|
|
||||||
|
2. **Agent-side `bottle_config:` override (the original #88
|
||||||
|
proposal).** Let an agent file carry an inline block that merges
|
||||||
|
over its referenced bottle. Ergonomically attractive — one file,
|
||||||
|
no second bottle — but it **breaks the trust boundary**: agent
|
||||||
|
files can come from `$CWD/.bot-bottle/agents/` in a cloned repo, so
|
||||||
|
a clone could redeclare egress routes, env mappings, and git
|
||||||
|
remotes — i.e. grant itself bottle-equivalent authority over
|
||||||
|
credentials and network egress. The home-only-bottle invariant
|
||||||
|
exists precisely to stop this.
|
||||||
|
|
||||||
|
3. **Bottle-side `extends:` (chosen).** Move composition to the
|
||||||
|
bottle layer, where it inherits the home-only property for free:
|
||||||
|
only `$HOME` bottles can declare `extends:`, and only `$HOME`
|
||||||
|
bottles can be its target. Identical duplication relief to option
|
||||||
|
2, none of its trust erosion. The cost is that an override requires
|
||||||
|
a (home-owned) child bottle rather than an inline agent block —
|
||||||
|
which is the *point*: the override authority stays in `$HOME`.
|
||||||
|
|
||||||
|
`extends:` wins because it solves the duplication pain entirely on the
|
||||||
|
trusted side of the agent-vs-bottle boundary. (PRD 0027 later lifts a
|
||||||
|
deliberately narrow, non-credential field — `git.user` — to the agent
|
||||||
|
layer, on the separate reasoning that commit identity is not a
|
||||||
|
capability; egress, credentials, and remotes stay bottle-only.)
|
||||||
|
|
||||||
## Goals / Success Criteria
|
## Goals / Success Criteria
|
||||||
|
|
||||||
- Add `extends: <bottle-name>` to the bottle frontmatter schema.
|
- Add `extends: <bottle-name>` to the bottle frontmatter schema.
|
||||||
@@ -58,9 +93,9 @@ bottle-equivalent config.
|
|||||||
|
|
||||||
## Non-goals
|
## Non-goals
|
||||||
|
|
||||||
- **No agent-side `bottle_config:`.** That's the design issue #88
|
- **No agent-side `bottle_config:`.** Option 2 under "Alternatives
|
||||||
considered and weighed against; this PRD is the alternative
|
considered" — weighed and rejected on trust grounds. Don't
|
||||||
picked in the issue's design discussion. Don't reintroduce it.
|
reintroduce it.
|
||||||
- **No additive list merges** (e.g., `routes: append` keyword).
|
- **No additive list merges** (e.g., `routes: append` keyword).
|
||||||
The `extends:` design uses full-replace for list-valued fields
|
The `extends:` design uses full-replace for list-valued fields
|
||||||
(see "Merge rules"); if a use case shows up that genuinely
|
(see "Merge rules"); if a use case shows up that genuinely
|
||||||
@@ -126,8 +161,7 @@ expectation. (Same model as shell `export` precedence.)
|
|||||||
`git.remotes` is also keyed, so it follows dict-style inheritance:
|
`git.remotes` is also keyed, so it follows dict-style inheritance:
|
||||||
children can override one host without restating every remote. The
|
children can override one host without restating every remote. The
|
||||||
remote entry is replaced as a whole on host collision because
|
remote entry is replaced as a whole on host collision because
|
||||||
`Upstream`, `IdentityFile`, `KnownHostKey`, and `ExtraHosts` are
|
`Upstream`, `IdentityFile`, and `KnownHostKey` are tightly coupled.
|
||||||
tightly coupled.
|
|
||||||
|
|
||||||
The `git.user` dataclass-overlay (each non-empty field wins
|
The `git.user` dataclass-overlay (each non-empty field wins
|
||||||
individually) is so a parent can declare `git.user.name` and a
|
individually) is so a parent can declare `git.user.name` and a
|
||||||
@@ -167,7 +201,7 @@ Bottles continue to be loaded from `$HOME/.bot-bottle/bottles/`
|
|||||||
only (`Manifest.from_md_dirs` is unchanged). The `extends:` field
|
only (`Manifest.from_md_dirs` is unchanged). The `extends:` field
|
||||||
references another file in that same directory. No cwd-readable
|
references another file in that same directory. No cwd-readable
|
||||||
file gains the ability to declare or modify bottle config — the
|
file gains the ability to declare or modify bottle config — the
|
||||||
attack surface from issue #88's comment thread stays closed.
|
attack surface from option 2 ("Alternatives considered") stays closed.
|
||||||
|
|
||||||
If a future change ever introduces cwd-loaded bottles, the
|
If a future change ever introduces cwd-loaded bottles, the
|
||||||
`extends:` resolver should be gated to forbid a `$CWD` bottle
|
`extends:` resolver should be gated to forbid a `$CWD` bottle
|
||||||
|
|||||||
@@ -86,7 +86,10 @@ agent_provider:
|
|||||||
|
|
||||||
## Open questions
|
## Open questions
|
||||||
|
|
||||||
- The initial Codex auth role is `codex_auth`; it provides a non-secret `OPENAI_API_KEY` placeholder to the agent while egress holds the real token.
|
- `codex_auth` is retained as a placeholder marker for follow-up Codex
|
||||||
|
credential-injection work. The Codex template should not inject an
|
||||||
|
`OPENAI_API_KEY` placeholder; Codex bottles use device/ChatGPT login
|
||||||
|
state instead.
|
||||||
- Existing state-folder transcript capture is Claude-specific and should remain gated to Claude until the follow-up state/transcript refactor.
|
- Existing state-folder transcript capture is Claude-specific and should remain gated to Claude until the follow-up state/transcript refactor.
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|||||||
@@ -0,0 +1,170 @@
|
|||||||
|
# PRD 0028: git-gate new-branch push scan scope
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-05-29
|
||||||
|
- **Issue:** #106
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
git-gate's pre-receive hook scans the **entire ancestry** of a *new*
|
||||||
|
branch for secrets, so any pre-existing finding in repo history blocks
|
||||||
|
every new-branch push. Scope the scan to the commits a push actually
|
||||||
|
introduces (`$new --not --all`) so a push is gated on what *it* adds,
|
||||||
|
not on what's already on the gate/upstream. Also harden the forward
|
||||||
|
`ssh` against hangs. Net: new branches can be pushed through the gate
|
||||||
|
again, with no loss of leak-detection coverage.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
In `git_gate_render_hook()` (`bot_bottle/git_gate.py`) the pre-receive
|
||||||
|
hook chooses the gitleaks revision range per ref:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
if [ "$old" = "$zero" ]; then
|
||||||
|
log_opts="$new" # new branch: `git log <new>` = the FULL ancestry
|
||||||
|
else
|
||||||
|
log_opts="$old..$new" # existing branch: just the pushed delta
|
||||||
|
fi
|
||||||
|
gitleaks git --log-opts="$log_opts" ... # exit 1 if ANY finding in range
|
||||||
|
```
|
||||||
|
|
||||||
|
For a **new** ref there is no `old` to diff against, so the hook passes
|
||||||
|
`$new`, which `git log` expands to every commit reachable from the new
|
||||||
|
tip. This repo's history contains 11 deliberately secret-shaped strings
|
||||||
|
(demo manifests, `docs/demo.tape`, and the pipelock/sandbox-escape
|
||||||
|
integration tests that exist *to exercise* the DLP). gitleaks reports
|
||||||
|
`438 commits scanned … leaks found: 11`, the hook `exit 1`s, and the
|
||||||
|
push is rejected. Confirmed live against issue #106's bottle: the
|
||||||
|
branch never lands in the bare repo and is never forwarded.
|
||||||
|
|
||||||
|
Consequence: **no new branch can ever be pushed through git-gate** as
|
||||||
|
long as a single historical finding exists — which is permanent.
|
||||||
|
|
||||||
|
Two adjacent problems surfaced while diagnosing #106:
|
||||||
|
|
||||||
|
- The rejection is **invisible to the client** — over the `git://` +
|
||||||
|
smolmachines forward it presented as a ~75s silent hang, not a
|
||||||
|
`remote: git-gate: gitleaks rejected …` message.
|
||||||
|
- The forward `ssh` lacks `BatchMode`/`ConnectTimeout`, so an
|
||||||
|
unreachable upstream or a prompt would hang the hook indefinitely.
|
||||||
|
(Not the cause of #106 — the forward itself works — but a latent
|
||||||
|
hang risk.)
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- A new-branch push is scanned **only for the commits it introduces**
|
||||||
|
(reachable from `$new`, not from any ref the gate already has).
|
||||||
|
- A new branch that adds no new findings **pushes successfully**, even
|
||||||
|
though historical fixtures still trip a full-history scan.
|
||||||
|
- A new branch that *does* introduce a finding is **still rejected**.
|
||||||
|
- No reduction in leak coverage for the commits a push actually brings
|
||||||
|
to the upstream (see "Security analysis").
|
||||||
|
- Forward `ssh` fails fast (`BatchMode=yes` + `ConnectTimeout`) instead
|
||||||
|
of hanging on a prompt/unreachable upstream.
|
||||||
|
- Existing git-gate unit + integration tests stay green; new tests lock
|
||||||
|
the scoped-scan behaviour.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- **Scrubbing the historical fixture findings.** They're intentional
|
||||||
|
test/demo inputs; scoping the scan resolves the practical problem
|
||||||
|
without rewriting history.
|
||||||
|
- **Relaxing the existing-branch path.** `$old..$new` already scans the
|
||||||
|
delta; this PRD only fixes the new-ref branch (and optionally unifies
|
||||||
|
on `--not --all`, see Open questions).
|
||||||
|
- **The client-visibility fix is investigation-gated.** Surfacing the
|
||||||
|
rejection over the `git://` + smolmachines path may need separate work
|
||||||
|
(sideband relay); tracked here but may land as a follow-up rather than
|
||||||
|
block the scan-scope fix.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Scoped scan
|
||||||
|
|
||||||
|
Replace the new-ref range with one that excludes everything the gate
|
||||||
|
already knows:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
if [ "$new" = "$zero" ]; then
|
||||||
|
continue # deletion: nothing to scan (unchanged)
|
||||||
|
elif [ "$old" = "$zero" ]; then
|
||||||
|
log_opts="$new --not --all" # new branch: only commits new to the gate
|
||||||
|
else
|
||||||
|
log_opts="$old..$new" # existing branch: the pushed delta
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
`git log $new --not --all` = commits reachable from the pushed tip but
|
||||||
|
**not** reachable from any ref already in the gate's bare repo.
|
||||||
|
|
||||||
|
### Security analysis (why excluding "already-on-the-gate" is safe)
|
||||||
|
|
||||||
|
Commits enter the gate's bare repo by exactly two paths:
|
||||||
|
|
||||||
|
1. **mirror-fetch from the upstream** — the bare repo is
|
||||||
|
`remote add --mirror=fetch origin`, and the access-hook fetches the
|
||||||
|
upstream before every upload-pack; and
|
||||||
|
2. **a push through the gate** — which is gitleaks-scanned before it is
|
||||||
|
forwarded.
|
||||||
|
|
||||||
|
Therefore every commit reachable from a gate ref is *already on the
|
||||||
|
upstream* or *was already scanned when pushed*. A commit excluded by
|
||||||
|
`--not --all` cannot be a new secret arriving at the upstream via this
|
||||||
|
push:
|
||||||
|
|
||||||
|
- if it's already upstream, re-scanning changes nothing — the content is
|
||||||
|
already there and blocking this branch wouldn't remove it; and
|
||||||
|
- if it arrived via an earlier push, it was already scanned.
|
||||||
|
|
||||||
|
The only commits that can carry a *new* secret upstream are the ones the
|
||||||
|
push introduces — exactly the set `$new --not --all` scans. An agent
|
||||||
|
cannot pre-seed a secret commit as "already known" to dodge the scan: it
|
||||||
|
can't write refs into the bare repo except by pushing (which scans), and
|
||||||
|
the mirror refs come only from the trusted upstream.
|
||||||
|
|
||||||
|
**Invariant this relies on:** the bare repo's refs are populated *only*
|
||||||
|
by upstream mirror-fetch and gitleaks-gated pushes. That holds in the
|
||||||
|
current design (nothing writes refs out-of-band); revisit if that
|
||||||
|
changes.
|
||||||
|
|
||||||
|
### Forward ssh hardening
|
||||||
|
|
||||||
|
Add `-o BatchMode=yes -o ConnectTimeout=<n>` to the hook's `ssh_cmd` so
|
||||||
|
a prompt or unreachable upstream fails fast with a clear error instead
|
||||||
|
of hanging the receive-pack.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
1. **PRD (this commit).**
|
||||||
|
2. **Hook scan scope + ssh hardening.** Edit `git_gate_render_hook()`:
|
||||||
|
the new-ref range → `$new --not --all`; add `BatchMode`/`ConnectTimeout`
|
||||||
|
to `ssh_cmd`. Unit tests in `test_git_gate.py` asserting the rendered
|
||||||
|
hook uses the scoped range for new refs and the hardened ssh flags.
|
||||||
|
3. **Integration coverage.** A new-branch push carrying no new finding
|
||||||
|
succeeds through a gate whose history contains a fixture finding; a
|
||||||
|
new-branch push that introduces a finding is still rejected.
|
||||||
|
4. **(Optional / follow-up) client visibility.** Make a gitleaks/forward
|
||||||
|
rejection reach the client as a `remote:` error over the git:// +
|
||||||
|
smolmachines path.
|
||||||
|
|
||||||
|
## Testing strategy
|
||||||
|
|
||||||
|
- **Unit (must):** rendered-hook assertions — new-ref uses
|
||||||
|
`$new --not --all`, existing-ref still `$old..$new`, deletion still
|
||||||
|
skipped; ssh_cmd carries `BatchMode=yes` + a `ConnectTimeout`.
|
||||||
|
- **Integration (should):** against a real gate seeded with a
|
||||||
|
fixture-bearing history, a clean new branch forwards to the upstream;
|
||||||
|
a new branch with a planted secret is rejected. Skips cleanly on hosts
|
||||||
|
that can't run the bundle (same shape as the existing git-gate
|
||||||
|
integration test).
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Unify both branches on `--not --all`?** It's also more robust than
|
||||||
|
`$old..$new` for non-fast-forward/force pushes (which can skip commits
|
||||||
|
off the direct path). Tempting to use it for the existing-ref case
|
||||||
|
too; deferred to keep this change tight, but worth a follow-up.
|
||||||
|
- **Client visibility mechanism.** Whether the silent-hang is a git
|
||||||
|
daemon sideband-relay issue or specific to the smolmachines forward
|
||||||
|
needs a focused repro before committing to a fix.
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
# PRD 0029: Provider auth credentials through egress
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-05-29
|
||||||
|
- **Issue:** #109
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Allow provider bottles to inject host credentials into the egress
|
||||||
|
sidecar without exposing them to the agent. Codex uses
|
||||||
|
`agent_provider.forward_host_credentials` for ChatGPT/device-login
|
||||||
|
access tokens. Claude uses `agent_provider.auth_token` to name the host
|
||||||
|
env var holding its OAuth token, which egress injects on
|
||||||
|
`api.anthropic.com` requests.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Codex bottles can reach OpenAI hosts after they are added to egress, but
|
||||||
|
requests to Codex's ChatGPT-backed API endpoints still fail with HTTP
|
||||||
|
403 when the egress route is unauthenticated. The egress proxy strips
|
||||||
|
agent-originated `Authorization` headers and only re-injects auth for
|
||||||
|
routes that declare an egress-owned token. Bare `api.openai.com` or
|
||||||
|
`chatgpt.com` routes therefore forward Codex requests without the
|
||||||
|
ChatGPT bearer token.
|
||||||
|
|
||||||
|
Copying the host `~/.codex/auth.json` into the agent would solve auth
|
||||||
|
mode detection but would also put access and refresh material inside the
|
||||||
|
agent sandbox. That cuts against bot-bottle's credential minimization
|
||||||
|
model: provider credentials should live in the sidecar boundary when
|
||||||
|
possible, not in the agent.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- A Codex bottle with host ChatGPT auth can call Codex's
|
||||||
|
`api.openai.com` and `chatgpt.com` endpoints through egress.
|
||||||
|
- Host credential forwarding happens only when the bottle declares
|
||||||
|
`agent_provider.forward_host_credentials: true`.
|
||||||
|
- The agent container does not receive `OPENAI_API_KEY`,
|
||||||
|
`CODEX_ACCESS_TOKEN`, or real `tokens.access_token` /
|
||||||
|
`tokens.refresh_token` values.
|
||||||
|
- The agent container receives only a dummy Codex `auth.json` that
|
||||||
|
preserves the host auth-mode shape, keeps the selected ChatGPT
|
||||||
|
account id, and replaces credential values with placeholders.
|
||||||
|
- Egress route files remain non-secret: they contain only host/path/auth
|
||||||
|
slot metadata, never token values.
|
||||||
|
- Missing, API-key, malformed, or expired host Codex auth fails
|
||||||
|
launch with a clear operator-facing message.
|
||||||
|
- Existing Claude OAuth placeholder behavior remains unchanged.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Refreshing Codex tokens in the sidecar. The first cut reads the host's
|
||||||
|
current access token at launch; operators can restart after host Codex
|
||||||
|
refreshes auth.
|
||||||
|
- Copying host `~/.codex/auth.json` credentials into the agent.
|
||||||
|
- Allowing arbitrary host credential forwarding beyond the two providers
|
||||||
|
covered here (Codex ChatGPT/device-login and Claude OAuth).
|
||||||
|
- Hot-applying new authenticated Codex routes to an existing running
|
||||||
|
sidecar. The current hot-apply path cannot safely populate new token
|
||||||
|
env slots in an already-running container.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
### In scope
|
||||||
|
|
||||||
|
- Add `agent_provider.forward_host_credentials` to the bottle manifest
|
||||||
|
schema, defaulting to `false`.
|
||||||
|
- Support the flag for `agent_provider.template: codex`.
|
||||||
|
- Add `agent_provider.auth_token` to the bottle manifest schema.
|
||||||
|
- Support the field for `agent_provider.template: claude`: the named
|
||||||
|
host env var is forwarded only into the egress sidecar as the Bearer
|
||||||
|
token for `api.anthropic.com`, and a placeholder
|
||||||
|
`CLAUDE_CODE_OAUTH_TOKEN` is set in the agent so the Claude Code CLI
|
||||||
|
starts without a real credential.
|
||||||
|
- Remove the `claude_code_oauth` egress route role, which previously
|
||||||
|
required operators to declare the OAuth route manually. The provisioner
|
||||||
|
now injects it from `auth_token`.
|
||||||
|
- Read host Codex auth from `$CODEX_HOME/auth.json` when `CODEX_HOME` is
|
||||||
|
set, otherwise from `~/.codex/auth.json`.
|
||||||
|
- Extract only `tokens.access_token` for egress injection.
|
||||||
|
- Generate a dummy agent-side `auth.json` from the host auth file's
|
||||||
|
mode and key shape, without copying real token values.
|
||||||
|
- Validate that host auth is not API-key mode and the access token is
|
||||||
|
present, JWT-shaped, and not expired.
|
||||||
|
- Add or upgrade `api.openai.com` and `chatgpt.com` egress routes to
|
||||||
|
inject that access token via a shared `EGRESS_TOKEN_N` sidecar env
|
||||||
|
slot.
|
||||||
|
- Pass the extracted token only into the sidecar compose/run
|
||||||
|
environment, alongside other egress token values.
|
||||||
|
|
||||||
|
### Out of scope
|
||||||
|
|
||||||
|
- Sidecar-owned refresh using `tokens.refresh_token`.
|
||||||
|
- Sharing full Codex auth state with the agent.
|
||||||
|
- Supporting host credential forwarding for non-Codex providers.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Manifest
|
||||||
|
|
||||||
|
Extend `agent_provider`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
agent_provider:
|
||||||
|
template: codex
|
||||||
|
forward_host_credentials: true
|
||||||
|
```
|
||||||
|
|
||||||
|
The field defaults to `false`. If set on a non-Codex provider, manifest
|
||||||
|
validation should reject it until that provider has a concrete,
|
||||||
|
credential-minimizing implementation.
|
||||||
|
|
||||||
|
### Host auth extraction
|
||||||
|
|
||||||
|
At prepare/launch time, when the flag is enabled for Codex:
|
||||||
|
|
||||||
|
1. Resolve the host Codex home directory from `$CODEX_HOME`, falling
|
||||||
|
back to `~/.codex`.
|
||||||
|
2. Parse `auth.json`.
|
||||||
|
3. Require user/device auth mode rather than API-key auth.
|
||||||
|
4. Require a non-empty `tokens.access_token`.
|
||||||
|
5. Parse the JWT payload enough to require an `exp` claim in the future.
|
||||||
|
6. Return only the access token value to the launch path.
|
||||||
|
|
||||||
|
Errors should name the missing or invalid condition and point the
|
||||||
|
operator at `codex login --device-auth`, without printing token values.
|
||||||
|
|
||||||
|
### Egress route
|
||||||
|
|
||||||
|
When forwarding host Codex credentials, the effective egress route table
|
||||||
|
should contain authenticated `api.openai.com` and `chatgpt.com` routes.
|
||||||
|
If the bottle already declares either host as a bare-pass route, upgrade
|
||||||
|
it in the effective route table rather than requiring a duplicate
|
||||||
|
manifest entry. If the bottle already declares an authenticated route for
|
||||||
|
either host, fail rather than guessing whether to override
|
||||||
|
operator-provided auth, unless that route already uses the synthetic
|
||||||
|
Codex host credential token reference.
|
||||||
|
|
||||||
|
The rendered route should look like any other egress-owned auth route:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
routes:
|
||||||
|
- host: "api.openai.com"
|
||||||
|
auth_scheme: "Bearer"
|
||||||
|
token_env: "EGRESS_TOKEN_N"
|
||||||
|
- host: "chatgpt.com"
|
||||||
|
auth_scheme: "Bearer"
|
||||||
|
token_env: "EGRESS_TOKEN_N"
|
||||||
|
```
|
||||||
|
|
||||||
|
The access token value is supplied through the sidecar process
|
||||||
|
environment for that `EGRESS_TOKEN_N` slot. It must not be written to
|
||||||
|
`routes.yaml`, compose files, env files, logs, or user-facing output.
|
||||||
|
|
||||||
|
### Data flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
H["Host ~/.codex/auth.json"] --> L["bot-bottle launch"]
|
||||||
|
L -->|access token only| S["egress sidecar env"]
|
||||||
|
L -->|dummy auth.json only| A
|
||||||
|
A["Codex agent"] -->|HTTPS via proxy, auth stripped| E["egress"]
|
||||||
|
E -->|Bearer injected from env| C["api.openai.com / chatgpt.com"]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
1. **PRD first.** Land this document as the first commit on the feature
|
||||||
|
branch.
|
||||||
|
2. **Manifest schema.** Add `forward_host_credentials`, validation, and
|
||||||
|
unit tests.
|
||||||
|
3. **Host Codex auth reader.** Add a small stdlib-only helper for
|
||||||
|
parsing and validating host Codex auth without printing values.
|
||||||
|
4. **Effective egress route.** Add/upgrade the Codex API routes when the
|
||||||
|
flag is enabled, and add tests for bare route upgrade,
|
||||||
|
missing-route insertion, and authenticated-route conflict.
|
||||||
|
5. **Agent auth marker.** Provision a dummy Codex `auth.json` into the
|
||||||
|
agent home so Codex selects the host's user/device auth branch while
|
||||||
|
real credentials stay in egress.
|
||||||
|
6. **Launch wiring.** Pass the host access token into the egress sidecar
|
||||||
|
env for Docker and smolmachines without exposing it to the agent.
|
||||||
|
7. **Docs and tests.** Update README examples and run the unit suite.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- Should a later version support sidecar refresh using the host refresh
|
||||||
|
token, or should restart-on-expiry remain the policy?
|
||||||
|
- Should telemetry hosts such as `ab.chatgpt.com` stay blocked by
|
||||||
|
default even when Codex ChatGPT auth is enabled?
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Gitea issue #109: Codex ChatGPT auth should inject host access token
|
||||||
|
via egress.
|
||||||
|
- PRD 0017: Egress-proxy — universal MITM with path filtering + auth
|
||||||
|
injection.
|
||||||
|
- PRD 0026: Agent provider templates.
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
# PRD 0030: Deduplicate egress token resolution across backends
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #118
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Eliminate the duplicated egress token resolution block — which resolved
|
||||||
|
manifest-declared tokens and the Codex host credential — by moving
|
||||||
|
provider-specific token reading into `AgentProvisionPlan.provisioned_env`
|
||||||
|
at prepare time, and having both backends merge that map into `os.environ`
|
||||||
|
before calling the now-generic `egress_resolve_token_values`.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The same logic block appeared in two places:
|
||||||
|
|
||||||
|
- `bot_bottle/backend/docker/launch.py` (~line 183): inline inside
|
||||||
|
`launch()`, building `token_values` before `compose_up`.
|
||||||
|
- `bot_bottle/backend/smolmachines/launch.py` (~line 422): the private
|
||||||
|
`_resolve_token_env` helper, called before `_bundle.start_bundle`.
|
||||||
|
|
||||||
|
Both blocks:
|
||||||
|
1. Short-circuit to `{}` when there are no egress routes.
|
||||||
|
2. Call `egress_resolve_token_values(token_env_map, host_env)` to
|
||||||
|
resolve ordinary manifest-declared token refs.
|
||||||
|
3. Check `agent_provider.forward_host_credentials` and, when true,
|
||||||
|
call `codex_host_access_token` and slot the result into any
|
||||||
|
`token_env` whose `token_ref` is `CODEX_HOST_CREDENTIAL_TOKEN_REF`.
|
||||||
|
|
||||||
|
The duplication means any change to step 3 must be applied twice.
|
||||||
|
PRD 0029, which introduced `forward_host_credentials`, already had to
|
||||||
|
wire both backends; the next change would too. This is a near-certain
|
||||||
|
future sync bug.
|
||||||
|
|
||||||
|
`egress_resolve_token_values` also carried a sentinel `continue` skip
|
||||||
|
for `CODEX_HOST_CREDENTIAL_TOKEN_REF`, which tied an otherwise generic
|
||||||
|
egress helper to a Codex-specific contract.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- The `forward_host_credentials` resolution logic exists in exactly one
|
||||||
|
place in the codebase.
|
||||||
|
- Both `docker/launch.py` and `smolmachines/launch.py` call
|
||||||
|
`egress_resolve_token_values` with no provider-specific arguments.
|
||||||
|
- `egress_resolve_token_values` is fully generic — it neither knows nor
|
||||||
|
cares about provider identity or the `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
||||||
|
sentinel.
|
||||||
|
- No behaviour change for either backend.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Changes to token resolution semantics. This is a pure refactor.
|
||||||
|
- Adding support for any new credential type or provider.
|
||||||
|
- Consolidating any other backend differences beyond this one block.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### `AgentProvisionPlan.provisioned_env`
|
||||||
|
|
||||||
|
Add `provisioned_env: dict[str, str]` (default empty) to
|
||||||
|
`AgentProvisionPlan`. This map holds host-side secrets that the
|
||||||
|
provisioning stage resolved and that egress needs injected into the
|
||||||
|
sidecar environ.
|
||||||
|
|
||||||
|
When `forward_host_credentials=True` for Codex, `agent_provision_plan`
|
||||||
|
calls `codex_host_access_token(host_env)` and stores the result under
|
||||||
|
`CODEX_HOST_CREDENTIAL_TOKEN_REF`. This is already the prepare-time
|
||||||
|
stage where `write_codex_dummy_auth_file` runs, so the access-token
|
||||||
|
read is colocated with all other Codex-specific provisioning.
|
||||||
|
|
||||||
|
### Backend call sites
|
||||||
|
|
||||||
|
Both backends merge `provisioned_env` over `os.environ` before calling
|
||||||
|
`egress_resolve_token_values`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||||
|
token_values = egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||||
|
```
|
||||||
|
|
||||||
|
`provisioned_env` values take precedence over same-named host env vars,
|
||||||
|
so the Codex token slot resolves from the map written at prepare time
|
||||||
|
rather than from a raw `os.environ` lookup.
|
||||||
|
|
||||||
|
### `egress_resolve_token_values`
|
||||||
|
|
||||||
|
Remove the `CODEX_HOST_CREDENTIAL_TOKEN_REF` sentinel `continue` skip.
|
||||||
|
The function now resolves every slot in `token_env_map` from `host_env`
|
||||||
|
without special-casing any key name. The `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
||||||
|
key is present in `effective_env` (injected by `provisioned_env`) exactly
|
||||||
|
when it is needed.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- `bot_bottle/agent_provider.py`: add `provisioned_env` field; populate
|
||||||
|
it for Codex when `forward_host_credentials=True`.
|
||||||
|
- `bot_bottle/egress.py`: remove sentinel skip; remove
|
||||||
|
`egress_resolve_token_values_with_provider` (the intermediate function
|
||||||
|
that was introduced and then superseded by this design); drop the
|
||||||
|
`codex_auth` import.
|
||||||
|
- `bot_bottle/backend/docker/launch.py`: replace the provider-aware
|
||||||
|
resolution block with the `effective_env` merge.
|
||||||
|
- `bot_bottle/backend/smolmachines/launch.py`: same replacement in
|
||||||
|
`_resolve_token_env`.
|
||||||
|
- `tests/unit/test_agent_provider.py`: add tests verifying
|
||||||
|
`provisioned_env` is populated (or empty) for Codex with and without
|
||||||
|
`forward_host_credentials`.
|
||||||
|
- `tests/unit/test_egress.py`: remove `TestResolveTokenValuesWithProvider`;
|
||||||
|
replace the sentinel-skip test with a test verifying the Codex token
|
||||||
|
ref resolves normally when present in `host_env`.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue #118: Deduplicate egress token resolution across backends.
|
||||||
|
- Issue #117: Complexity hotspots in launch, egress, and auth paths
|
||||||
|
(source of the finding).
|
||||||
|
- PRD 0029: Provider auth credentials through egress (introduced the
|
||||||
|
duplicated block).
|
||||||
@@ -0,0 +1,241 @@
|
|||||||
|
# PRD 0031: Simplify egress route merge and consolidate Route types
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #120
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Replace `_merge_provider_route`'s five-case nested conditional with a
|
||||||
|
flat provisioned-wins merge, and make the mapping between the host-side
|
||||||
|
`EgressRoute` and the addon's `Route` explicit in one place. Covers the
|
||||||
|
two remaining open tasks from the #117 hotspot review.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
### 1. `_merge_provider_route` branching
|
||||||
|
|
||||||
|
`_merge_provider_route` in `bot_bottle/egress.py` handles five distinct
|
||||||
|
cases in a single function with interleaved conditions:
|
||||||
|
|
||||||
|
1. **append-new** — host not in manifest; append a fresh route.
|
||||||
|
2. **upgrade-bare** — host found, no existing auth; adopt provider auth.
|
||||||
|
3. **no-op** — host found, same auth; return unchanged.
|
||||||
|
4. **tls-passthrough upgrade** — same as no-op but provider sets
|
||||||
|
`tls_passthrough=True`; flip the flag on the existing route.
|
||||||
|
5. **conflict-die** — host found, different auth; hard error.
|
||||||
|
|
||||||
|
Cases 3 and 4 share a block with no-op as the invisible fall-through.
|
||||||
|
`_find_or_alloc_token_env` is duplicated between cases 2 and 1. In-place
|
||||||
|
replacements spell out every `EgressRoute` field explicitly, so a new
|
||||||
|
field added to the dataclass silently drops its value in any replacement
|
||||||
|
site that wasn't updated.
|
||||||
|
|
||||||
|
The root cause of the complexity is that the current merge tries to be
|
||||||
|
cooperative: it lets manifest routes coexist with provider routes and
|
||||||
|
attempts to upgrade bare manifest entries. This makes sense if the
|
||||||
|
manifest is authoritative, but the actual intended hierarchy is the
|
||||||
|
opposite — provider routes claim their hosts outright and the manifest
|
||||||
|
fills in what's left.
|
||||||
|
|
||||||
|
### 2. Three-way Route type fragmentation
|
||||||
|
|
||||||
|
`EgressRoute` (in `egress.py`) and `egress_addon_core.Route` are
|
||||||
|
separate dataclasses with overlapping but not identical field sets:
|
||||||
|
|
||||||
|
| Field | `EgressRoute` | addon `Route` |
|
||||||
|
|---|---|---|
|
||||||
|
| `host` | ✓ | ✓ |
|
||||||
|
| `path_allowlist` | ✓ | ✓ |
|
||||||
|
| `auth_scheme` | ✓ | ✓ |
|
||||||
|
| `token_env` | ✓ | ✓ |
|
||||||
|
| `token_ref` | ✓ (host-side) | — |
|
||||||
|
| `roles` | ✓ (host-side) | — |
|
||||||
|
| `tls_passthrough` | ✓ (pipelock concern) | — |
|
||||||
|
|
||||||
|
`egress_render_routes` serialises `EgressRoute` fields to YAML; the
|
||||||
|
addon's `load_routes` deserialises that YAML into `Route` objects. If a
|
||||||
|
field is added to `EgressRoute` that should appear in the YAML, both
|
||||||
|
`egress_render_routes` and `_parse_one` must be updated consistently.
|
||||||
|
The render function spells the field list out inline with no reference
|
||||||
|
to the addon's parser, so divergence is silent until runtime.
|
||||||
|
|
||||||
|
`egress_addon_core.Route` cannot be replaced by `EgressRoute` — the
|
||||||
|
addon file is copied flat into the sidecar container image (`/app/`) and
|
||||||
|
has no access to the `bot_bottle` package. The types must stay separate;
|
||||||
|
the risk is that they drift.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `egress_routes_for_bottle` implements a flat provisioned-wins merge:
|
||||||
|
provider routes claim their hosts; manifest routes for unclaimed hosts
|
||||||
|
append. No upgrade logic, no conflict detection.
|
||||||
|
- Token slot assignment is a single pass over the merged list, not
|
||||||
|
interleaved with the merge.
|
||||||
|
- `egress_render_routes` uses a single `_route_to_yaml_fields` helper
|
||||||
|
that explicitly lists the addon-visible fields, creating one place
|
||||||
|
where the `EgressRoute`→`Route` mapping is spelled out.
|
||||||
|
- All existing `TestProviderRouteMerge` and `TestRenderRoutes` tests
|
||||||
|
pass (adjusting assertions for any semantics changes described below).
|
||||||
|
- No behaviour change for existing manifests that don't trigger the
|
||||||
|
conflict-die or upgrade-bare paths.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Merging `EgressRoute` and `egress_addon_core.Route` into one class
|
||||||
|
(impossible: addon runs in a stdlib-only container environment).
|
||||||
|
- Changing what the addon does with a route once it has one.
|
||||||
|
- Changing `decide()` or `is_git_push_request()` in `egress_addon_core`
|
||||||
|
— those are already pure functions with good separation.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Merge: provisioned wins
|
||||||
|
|
||||||
|
The new hierarchy: **provisioned routes own their hosts; manifest routes
|
||||||
|
fill the gaps.**
|
||||||
|
|
||||||
|
```
|
||||||
|
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||||
|
|
||||||
|
effective = list(provider_routes)
|
||||||
|
effective += [r for r in manifest_routes if r.host.lower() not in provisioned_hosts]
|
||||||
|
```
|
||||||
|
|
||||||
|
Token slot assignment runs as a final pass over `effective` in order:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _assign_token_slots(
|
||||||
|
routes: list[EgressRoute],
|
||||||
|
) -> tuple[EgressRoute, ...]:
|
||||||
|
slot_for_ref: dict[str, str] = {}
|
||||||
|
out: list[EgressRoute] = []
|
||||||
|
for r in routes:
|
||||||
|
if r.auth_scheme and r.token_ref and not r.token_env:
|
||||||
|
token_env = slot_for_ref.get(r.token_ref)
|
||||||
|
if token_env is None:
|
||||||
|
token_env = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
||||||
|
slot_for_ref[r.token_ref] = token_env
|
||||||
|
r = dataclasses.replace(r, token_env=token_env)
|
||||||
|
out.append(r)
|
||||||
|
return tuple(out)
|
||||||
|
```
|
||||||
|
|
||||||
|
This replaces `_merge_provider_route`, `_find_or_alloc_token_env`, and
|
||||||
|
the slot-assignment loop inside `egress_manifest_routes`.
|
||||||
|
|
||||||
|
#### Semantics change
|
||||||
|
|
||||||
|
Under the old design, a manifest route for a provisioned host with a
|
||||||
|
*different* `auth_scheme` or `token_ref` raised a hard error. Under
|
||||||
|
provisioned-wins, the manifest entry is silently dropped. Operators who
|
||||||
|
relied on the conflict error to catch misconfigurations should audit
|
||||||
|
their manifests, but in practice this path was only reachable when a
|
||||||
|
manifest declared auth for `api.openai.com` or `chatgpt.com` with a
|
||||||
|
token ref other than `CODEX_HOST_CREDENTIAL_TOKEN_REF` while also
|
||||||
|
enabling `forward_host_credentials` — an unlikely combination.
|
||||||
|
|
||||||
|
Similarly, the "upgrade-bare" path (provider adopts a bare manifest
|
||||||
|
route's `path_allowlist`) is dropped: a provisioned host takes the
|
||||||
|
provider route's fields wholesale, and the manifest's `path_allowlist`
|
||||||
|
for that host is ignored.
|
||||||
|
|
||||||
|
### Route type mapping: `_route_to_yaml_fields`
|
||||||
|
|
||||||
|
Add a pure function in `egress.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _route_to_yaml_fields(r: EgressRoute) -> dict:
|
||||||
|
"""Return the addon-visible fields for one route.
|
||||||
|
|
||||||
|
This is the single authoritative mapping between `EgressRoute`
|
||||||
|
(host-side) and `egress_addon_core.Route` (sidecar-side). If a
|
||||||
|
field is added to `Route` that must appear in the YAML, add it
|
||||||
|
here and in `egress_addon_core._parse_one` together."""
|
||||||
|
fields: dict = {"host": r.host}
|
||||||
|
if r.auth_scheme and r.token_env:
|
||||||
|
fields["auth_scheme"] = r.auth_scheme
|
||||||
|
fields["token_env"] = r.token_env
|
||||||
|
if r.path_allowlist:
|
||||||
|
fields["path_allowlist"] = list(r.path_allowlist)
|
||||||
|
return fields
|
||||||
|
```
|
||||||
|
|
||||||
|
`egress_render_routes` delegates to it:
|
||||||
|
|
||||||
|
```python
|
||||||
|
for r in routes:
|
||||||
|
f = _route_to_yaml_fields(r)
|
||||||
|
lines.append(f' - host: "{f["host"]}"')
|
||||||
|
if "auth_scheme" in f:
|
||||||
|
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||||
|
lines.append(f' token_env: "{f["token_env"]}"')
|
||||||
|
if "path_allowlist" in f:
|
||||||
|
lines.append(" path_allowlist:")
|
||||||
|
for p in f["path_allowlist"]:
|
||||||
|
lines.append(f' - "{p}"')
|
||||||
|
```
|
||||||
|
|
||||||
|
The docstring on `_route_to_yaml_fields` is the explicit callout to
|
||||||
|
update both it and `_parse_one` together when the schema changes.
|
||||||
|
|
||||||
|
### `egress_manifest_routes` / `egress_routes_for_bottle`
|
||||||
|
|
||||||
|
`egress_manifest_routes` becomes a pure lifter with no slot assignment:
|
||||||
|
it reads each manifest route entry and returns an `EgressRoute` with
|
||||||
|
`token_env=""` (the slot to be filled later). The function's docstring
|
||||||
|
currently promises slot assignment; that promise moves to
|
||||||
|
`egress_routes_for_bottle`.
|
||||||
|
|
||||||
|
`egress_routes_for_bottle` becomes:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def egress_routes_for_bottle(
|
||||||
|
bottle: Bottle,
|
||||||
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
|
) -> tuple[EgressRoute, ...]:
|
||||||
|
manifest = egress_manifest_routes(bottle)
|
||||||
|
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||||
|
merged = list(provider_routes) + [
|
||||||
|
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||||
|
]
|
||||||
|
return _assign_token_slots(merged)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mitmproxy request logic
|
||||||
|
|
||||||
|
`egress_addon_core.decide()` and `is_git_push_request()` are already
|
||||||
|
pure functions; `egress_addon.EgressAddon.request()` is the minimal
|
||||||
|
mitmproxy glue (read host/path/headers from flow → call pure functions
|
||||||
|
→ apply result to flow). This split is already clean and requires no
|
||||||
|
structural change in this PRD.
|
||||||
|
|
||||||
|
## Test impact
|
||||||
|
|
||||||
|
- **`TestProviderRouteMerge`**: the `test_provider_route_upgrades_bare_manifest_route`
|
||||||
|
test asserts that a provider route preserves a bare manifest route's
|
||||||
|
`path_allowlist`. Under provisioned-wins that `path_allowlist` is
|
||||||
|
dropped. Update the test to reflect the new semantics.
|
||||||
|
- **`test_provider_route_conflicts_with_different_authed_manifest_route`**:
|
||||||
|
the conflict-die case no longer exists. Remove this test.
|
||||||
|
- All other merge and render tests should pass without modification.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
1. **PRD (this commit).** Sets the design.
|
||||||
|
2. **Merge refactor.** Replace `_merge_provider_route` and
|
||||||
|
`_find_or_alloc_token_env` with `_assign_token_slots` and the
|
||||||
|
flat provisioned-wins logic in `egress_routes_for_bottle`. Strip
|
||||||
|
slot assignment from `egress_manifest_routes`.
|
||||||
|
3. **Render consolidation.** Add `_route_to_yaml_fields`; update
|
||||||
|
`egress_render_routes` to use it.
|
||||||
|
4. **Test updates.** Adjust `TestProviderRouteMerge` for the semantics
|
||||||
|
changes above; confirm all render tests pass.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue #120: Refactor `_merge_provider_route` (expanded to include
|
||||||
|
Route type fragmentation).
|
||||||
|
- Issue #117: Complexity hotspots — source of both findings.
|
||||||
|
- PRD 0030: Deduplicate egress token resolution (prior egress cleanup).
|
||||||
@@ -0,0 +1,221 @@
|
|||||||
|
# PRD 0032: Decompose smolmachines launch and harden bringup sequencing
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #122
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Split `launch()` into named per-step helpers, replace the empirical
|
||||||
|
`time.sleep(1.5)` with a readiness poll, and file-lock loopback alias
|
||||||
|
allocation. Addresses the three actionable issues from the #117 hotspot
|
||||||
|
review of `smolmachines/launch.py`.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
### 1. `launch()` step ordering
|
||||||
|
|
||||||
|
`launch()` in `smolmachines/launch.py` is 207 lines. Seven sequenced
|
||||||
|
steps are marked by numbered inline comments (`# 1. Reserve a loopback
|
||||||
|
alias`, `# 2. Mint per-bottle CAs`, ...) — the sequencing is
|
||||||
|
load-bearing (CA paths must be filled before the bundle spec is built;
|
||||||
|
the bundle must be running before port discovery; the VM must be created
|
||||||
|
before the allowlist is patched), but the dependencies are enforced only
|
||||||
|
by linear ordering within one function. Adding a new daemon, changing
|
||||||
|
the port-forward strategy, or debugging a bringup failure requires
|
||||||
|
reading the whole function to understand what state each step produces.
|
||||||
|
Each step is also not individually testable without mocking the entire
|
||||||
|
surrounding context.
|
||||||
|
|
||||||
|
### 2. `time.sleep(1.5)` for libkrun exec-channel race
|
||||||
|
|
||||||
|
After `machine_start`, back-to-back `machine_exec` calls occasionally
|
||||||
|
hit a SIGKILL in libkrun's exec channel at ~100ms. The sleep is
|
||||||
|
documented as "1.5s is empirically enough; provisioning already takes
|
||||||
|
seconds so the wait is amortized." The failure mode if the sleep is
|
||||||
|
insufficient: the filesystem-repair exec (`chown -R node:node /home/node`)
|
||||||
|
is SIGKILLed silently, and the agent later bails with `ENOENT`/`EPERM`
|
||||||
|
when Claude Code tries to write to `~/.claude.json`. A poll-until-ready
|
||||||
|
loop is more robust than a fixed duration: it exits as soon as the exec
|
||||||
|
channel is up, fails loudly with a timeout if the VM never becomes
|
||||||
|
responsive, and is self-documenting about what it is waiting for.
|
||||||
|
|
||||||
|
### 3. Loopback alias allocation is not concurrent-safe
|
||||||
|
|
||||||
|
`loopback_alias.allocate()` reads docker container state to determine
|
||||||
|
which aliases are already in use, then returns the lowest free alias.
|
||||||
|
There is no lock between that read and the bundle's `docker run` (which
|
||||||
|
creates the container that will appear in future `docker ps` output). Two
|
||||||
|
simultaneous bottle launches can both see the same alias as free and
|
||||||
|
claim it, causing both bundles to bind on the same loopback IP. On macOS,
|
||||||
|
where users occasionally start multiple agents in quick succession, this
|
||||||
|
is a realistic failure mode.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Removing `force_allowlist` / the `--allow-cidr` DB patch. That is a
|
||||||
|
workaround for a smolvm 0.8.0 bug; removal is a one-liner when smolvm
|
||||||
|
honors the CLI flag upstream.
|
||||||
|
- Changing the ephemeral registry / crane detour in `local_registry.py`.
|
||||||
|
Required by Docker Desktop's network topology.
|
||||||
|
- Changing `_ensure_smolmachine`'s cache design. Cache invalidation by
|
||||||
|
docker image ID works; issue #111 tracks a separate stale-sidecar
|
||||||
|
concern.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### 1. Decompose `launch()` into named helpers
|
||||||
|
|
||||||
|
Extract six focused helpers. `launch()` becomes a coordinator that calls
|
||||||
|
them in order, passing the `ExitStack` for teardown registration:
|
||||||
|
|
||||||
|
```
|
||||||
|
_allocate_resources(plan, stack) → (loopback_ip, network)
|
||||||
|
```
|
||||||
|
Reserve the loopback alias, create the docker bridge network, register
|
||||||
|
teardown callbacks for both.
|
||||||
|
|
||||||
|
```
|
||||||
|
_mint_certs(plan) → plan
|
||||||
|
```
|
||||||
|
Pipelock TLS init (always). Egress TLS init when `plan.egress_plan.routes`
|
||||||
|
is non-empty. Returns the plan with CA paths filled via
|
||||||
|
`dataclasses.replace`.
|
||||||
|
|
||||||
|
```
|
||||||
|
_start_bundle(plan, network, loopback_ip, stack) → plan
|
||||||
|
```
|
||||||
|
Build the `BundleLaunchSpec`, resolve token env, start the bundle
|
||||||
|
container, register teardown. Returns the plan with `bundle_spec` updated
|
||||||
|
(or unchanged if no plan field carries it — callers consume `bundle_spec`
|
||||||
|
directly from this call's return value if needed).
|
||||||
|
|
||||||
|
```
|
||||||
|
_discover_urls(plan, loopback_ip) → plan
|
||||||
|
```
|
||||||
|
Look up host-side ports for the published container ports; assemble
|
||||||
|
`agent_proxy_url`, `agent_git_gate_host`, `agent_supervise_url`; stamp
|
||||||
|
them onto the plan and into `guest_env`.
|
||||||
|
|
||||||
|
```
|
||||||
|
_launch_vm(plan, agent_from_path, stack) → None
|
||||||
|
```
|
||||||
|
`machine_create` + `force_allowlist` + `machine_start`. Register
|
||||||
|
`machine_stop` and `machine_delete` teardown callbacks on the stack.
|
||||||
|
|
||||||
|
```
|
||||||
|
_init_vm(plan) → None
|
||||||
|
```
|
||||||
|
Filesystem-repair exec (`chown`/`chmod`) followed by
|
||||||
|
`_wait_exec_ready()`.
|
||||||
|
|
||||||
|
`launch()` reduces to:
|
||||||
|
|
||||||
|
```python
|
||||||
|
loopback_ip, network = _allocate_resources(plan, stack)
|
||||||
|
plan = _mint_certs(plan)
|
||||||
|
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||||
|
plan = _discover_urls(plan, loopback_ip)
|
||||||
|
agent_from_path = _ensure_smolmachine(plan.agent_image_ref,
|
||||||
|
dockerfile=plan.agent_dockerfile_path)
|
||||||
|
_launch_vm(plan, agent_from_path, stack)
|
||||||
|
_init_vm(plan)
|
||||||
|
prompt_path = provision(plan, plan.machine_name)
|
||||||
|
yield SmolmachinesBottle(...)
|
||||||
|
```
|
||||||
|
|
||||||
|
Each helper's inputs and outputs are explicit; each is independently
|
||||||
|
testable with a minimal set of mocks.
|
||||||
|
|
||||||
|
### 2. Replace `time.sleep(1.5)` with `_wait_exec_ready`
|
||||||
|
|
||||||
|
Add to `smolvm.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
||||||
|
"""Poll until `machine exec true` exits 0 or `timeout` elapses.
|
||||||
|
Replaces a fixed sleep after machine_start for the libkrun
|
||||||
|
exec-channel warm-up race."""
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
delay = 0.1
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
r = machine_exec(name, ["true"])
|
||||||
|
if r.returncode == 0:
|
||||||
|
return
|
||||||
|
remaining = deadline - time.monotonic()
|
||||||
|
if remaining <= 0:
|
||||||
|
break
|
||||||
|
time.sleep(min(delay, remaining))
|
||||||
|
delay = min(delay * 2, 0.5)
|
||||||
|
die(
|
||||||
|
f"smolvm machine {name!r}: exec channel not ready after "
|
||||||
|
f"{timeout:.0f}s — VM may have failed to boot."
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
`_init_vm` calls `wait_exec_ready` after the chown/chmod exec instead of
|
||||||
|
`time.sleep(1.5)`. The `time` import in `launch.py` is removed.
|
||||||
|
|
||||||
|
### 3. File-lock loopback alias allocation
|
||||||
|
|
||||||
|
Add to `loopback_alias.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import fcntl
|
||||||
|
|
||||||
|
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
||||||
|
|
||||||
|
def allocate(slug: str) -> str:
|
||||||
|
if not _is_macos():
|
||||||
|
return "127.0.0.1"
|
||||||
|
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
||||||
|
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||||
|
return _allocate_locked(slug)
|
||||||
|
|
||||||
|
def _allocate_locked(slug: str) -> str:
|
||||||
|
in_use = _aliases_in_use()
|
||||||
|
for ip in _pool_addresses():
|
||||||
|
if ip not in in_use:
|
||||||
|
return ip
|
||||||
|
die(...)
|
||||||
|
return ""
|
||||||
|
```
|
||||||
|
|
||||||
|
The lock is held only for the duration of `_aliases_in_use()` + the
|
||||||
|
`allocate` return. The bundle's `docker run` runs after the lock is
|
||||||
|
released. This is sufficient: once `docker run` returns, the container
|
||||||
|
is visible in docker state and future `allocate()` calls will see it.
|
||||||
|
The remaining window (lock released → container appears in docker state)
|
||||||
|
is narrowed from "the entire bringup sequence" to "a single subprocess
|
||||||
|
call," making a collision between two concurrent launches effectively
|
||||||
|
impossible in practice.
|
||||||
|
|
||||||
|
The lock is a no-op on Linux (the `_is_macos()` early-return fires
|
||||||
|
before the lock path is opened).
|
||||||
|
|
||||||
|
## Test impact
|
||||||
|
|
||||||
|
- Unit tests for each extracted helper can mock one subprocess boundary
|
||||||
|
at a time (smolvm, docker, pipelock TLS init) without wiring the full
|
||||||
|
`launch()` ExitStack.
|
||||||
|
- `wait_exec_ready` needs a test with `machine_exec` stubbed to return
|
||||||
|
non-zero N times before 0 — verifies the backoff loop and the timeout
|
||||||
|
die path.
|
||||||
|
- `allocate` tests are unchanged in shape; the lock is acquired and
|
||||||
|
released within the call so tests don't need to be aware of it.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
1. **PRD (this commit).** Sets the design.
|
||||||
|
2. **Decompose `launch()`.**
|
||||||
|
3. **Replace sleep with `wait_exec_ready`.**
|
||||||
|
4. **File-lock `allocate()`.**
|
||||||
|
5. **Tests.** Unit tests for each helper; `wait_exec_ready` backoff + timeout.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue #122: Decompose smolmachines launch and harden bringup sequencing.
|
||||||
|
- Issue #117: Complexity hotspots — source of the smolmachines/launch.py finding.
|
||||||
|
- Issue #111: Smolmachine sidecar doesn't reliably get refreshed (separate, not addressed here).
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
# PRD 0033: Manifest Schema Boundaries
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #125
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Split the manifest loader's schema validation, filesystem loading, `extends:`
|
||||||
|
resolution, and compatibility passthrough policy into named internal boundaries
|
||||||
|
without changing the public manifest format. The goal is to make
|
||||||
|
`bot_bottle/manifest.py` cheaper to extend and review while preserving the
|
||||||
|
strict validation behavior that keeps manifest mistakes visible.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/manifest.py` has become a broad schema surface. It owns dataclass
|
||||||
|
models, per-field validators, per-section unknown-key policy, Markdown
|
||||||
|
frontmatter loading, two-pass bottle inheritance, merge semantics, and
|
||||||
|
effective agent-to-bottle overlays in one file. The logic is deterministic and
|
||||||
|
well covered, but the number of concerns makes schema changes expensive:
|
||||||
|
reviewers have to re-derive loader behavior, parse-time validation, and
|
||||||
|
post-parse composition rules together.
|
||||||
|
|
||||||
|
One specific coupling is especially easy to miss: agent Markdown files are
|
||||||
|
allowed to double as Claude Code subagent files, so the manifest parser accepts
|
||||||
|
and ignores Claude Code frontmatter fields such as `name`, `description`,
|
||||||
|
`model`, `color`, and `memory`. That compatibility rule is encoded as a
|
||||||
|
passthrough allowlist alongside bot-bottle's own agent schema. If Claude Code
|
||||||
|
adds a frontmatter field and users start sharing files between
|
||||||
|
`~/.claude/agents/` and `.bot-bottle/agents/`, bot-bottle raises
|
||||||
|
`ManifestError` until the local passthrough policy is updated.
|
||||||
|
|
||||||
|
The current shape is workable, but it creates unnecessary risk for future
|
||||||
|
manifest features. A new field can accidentally mix parsing, inheritance, and
|
||||||
|
compatibility concerns in the same edit, or update one entry path
|
||||||
|
(`from_json_obj`) without matching the Markdown path (`from_md_dirs`).
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Preserve the existing public manifest schema and runtime behavior.
|
||||||
|
- Keep `Manifest`, `Bottle`, `Agent`, `GitEntry`, `GitUser`, `AgentProvider`,
|
||||||
|
`EgressRoute`, `EgressConfig`, and `PipelockRoutePolicy` import-compatible
|
||||||
|
from `bot_bottle.manifest`.
|
||||||
|
- Move Markdown file discovery and frontmatter loading behind a small internal
|
||||||
|
loader boundary with tests that show `$HOME` bottles, `$HOME` agents, `$CWD`
|
||||||
|
agent overrides, and ignored `$CWD` bottles still behave as before.
|
||||||
|
- Move bottle `extends:` resolution and merge rules behind a named internal
|
||||||
|
resolver boundary with tests for inheritance, replacement, cycle detection,
|
||||||
|
missing parents, and per-field `git.user` overlays.
|
||||||
|
- Centralize top-level allowed-key policy for bottle and agent schemas so
|
||||||
|
unknown-key errors remain strict and the allowed set is visible in one place
|
||||||
|
per schema.
|
||||||
|
- Make Claude Code passthrough fields a named compatibility policy with focused
|
||||||
|
tests that distinguish accepted passthrough keys from bot-bottle schema keys
|
||||||
|
and true typos.
|
||||||
|
- Keep both entry points, `Manifest.from_json_obj` and
|
||||||
|
`Manifest.from_md_dirs`, covered by tests for shared validation and shared
|
||||||
|
inheritance behavior.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No manifest format changes.
|
||||||
|
- No migration away from Markdown frontmatter or the stdlib-only YAML subset
|
||||||
|
parser.
|
||||||
|
- No dependency on Pydantic, PyYAML, JSON Schema, or another schema framework.
|
||||||
|
- No relaxation of strict unknown-key validation for bot-bottle fields.
|
||||||
|
- No provider-specific workspace, auth, launch, or egress changes.
|
||||||
|
- No user-facing CLI behavior changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- Internal module organization for manifest loading and composition.
|
||||||
|
- Validator helpers or schema-policy helpers that reduce duplicated
|
||||||
|
unknown-key and type-checking logic.
|
||||||
|
- Focused regression tests around the two existing load paths.
|
||||||
|
- Documentation comments that clarify compatibility policy where it is encoded.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Renaming public dataclass fields or changing their capitalization.
|
||||||
|
- Reworking callers outside the manifest boundary except for import updates
|
||||||
|
that are mechanically required by an internal split.
|
||||||
|
- Adding new manifest fields.
|
||||||
|
- Changing how `bot-bottle.json` legacy-file errors are reported.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Keep `bot_bottle.manifest` as the public facade. Existing imports should
|
||||||
|
continue to work from that module, even if implementation moves into internal
|
||||||
|
modules such as:
|
||||||
|
|
||||||
|
- `bot_bottle/manifest_model.py` for dataclasses and field-level parsing.
|
||||||
|
- `bot_bottle/manifest_loader.py` for filesystem layout, Markdown
|
||||||
|
frontmatter loading, stale legacy-file checks, and `$CWD` override rules.
|
||||||
|
- `bot_bottle/manifest_extends.py` for raw-bottle inheritance, cycle checks,
|
||||||
|
and merge semantics.
|
||||||
|
- `bot_bottle/manifest_schema.py` for allowed-key sets, passthrough policy,
|
||||||
|
and small validation helpers.
|
||||||
|
|
||||||
|
The exact filenames are not required. The required boundary is conceptual:
|
||||||
|
raw input loading, schema validation, bottle inheritance, and effective
|
||||||
|
agent-to-bottle overlays should be separable when reading and testing the code.
|
||||||
|
|
||||||
|
`Manifest.from_json_obj` should continue to accept a raw JSON-like dict and
|
||||||
|
feed the same raw bottle resolver used by Markdown loading. `Manifest.from_md_dirs`
|
||||||
|
should perform only filesystem discovery and Markdown parsing before passing
|
||||||
|
the same raw sections into the same validator/composer path. That shared path
|
||||||
|
prevents a future schema field from working in one entry point but not the
|
||||||
|
other.
|
||||||
|
|
||||||
|
Claude Code passthrough fields should be represented as an explicit
|
||||||
|
compatibility allowlist, named as such, and documented near the agent schema
|
||||||
|
policy. The parser should still ignore those fields after validation. Tests
|
||||||
|
should cover every passthrough field currently accepted and at least one
|
||||||
|
unknown field that remains an error.
|
||||||
|
|
||||||
|
The `extends:` resolver should remain raw-dict based until after inheritance is
|
||||||
|
resolved. Merge rules stay unchanged:
|
||||||
|
|
||||||
|
- scalar fields use child value when present.
|
||||||
|
- `env` merges by key with child values winning.
|
||||||
|
- `git.remotes` merges by upstream host, with child entries replacing duplicate
|
||||||
|
hosts and explicit empty maps clearing inherited remotes.
|
||||||
|
- `git.user` overlays per field.
|
||||||
|
- `egress` remains full-replace when declared by the child.
|
||||||
|
- cycles, missing parents, and self-reference remain `ManifestError`s.
|
||||||
|
|
||||||
|
## Implementation Chunks
|
||||||
|
|
||||||
|
1. Add focused characterization tests for agent allowed keys, Claude Code
|
||||||
|
passthrough fields, and parity between `from_json_obj` and Markdown loading.
|
||||||
|
2. Extract allowed-key and compatibility policy helpers while keeping
|
||||||
|
`bot_bottle.manifest` as the import surface.
|
||||||
|
3. Extract raw Markdown loading into a loader boundary and rerun existing
|
||||||
|
PRD 0011 tests unchanged.
|
||||||
|
4. Extract bottle inheritance and merge rules into a resolver boundary and
|
||||||
|
rerun existing PRD 0025 tests unchanged.
|
||||||
|
5. Trim `bot_bottle.manifest` to the public facade and model composition,
|
||||||
|
leaving compatibility imports for existing callers.
|
||||||
|
|
||||||
|
Each chunk should be mergeable on its own and should keep the test suite green.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
Run the existing manifest-focused unit tests after each chunk:
|
||||||
|
|
||||||
|
- `tests/unit/test_manifest_md_load.py`
|
||||||
|
- `tests/unit/test_manifest_extends.py`
|
||||||
|
- `tests/unit/test_manifest_git.py`
|
||||||
|
- `tests/unit/test_manifest_git_user.py`
|
||||||
|
- `tests/unit/test_manifest_agent_git_user.py`
|
||||||
|
- `tests/unit/test_manifest_egress.py`
|
||||||
|
- `tests/unit/test_manifest_runtime.py`
|
||||||
|
|
||||||
|
Add new tests only where they lock down boundary behavior not already covered,
|
||||||
|
especially compatibility passthrough and entry-point parity.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- Should the Claude Code passthrough allowlist intentionally track a documented
|
||||||
|
upstream schema, or should bot-bottle keep a narrow local allowlist and update
|
||||||
|
it only when users need a new shared-file field?
|
||||||
|
- Should the public facade continue exposing every helper that tests currently
|
||||||
|
import from `bot_bottle.manifest`, or should tests move to public behavior
|
||||||
|
only during this cleanup?
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
# PRD 0034: Sidecar Restart and Shutdown Semantics
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #126
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make the sidecar bundle supervisor's signal, restart, and exit-code behavior
|
||||||
|
explicit and easier to reason about. In particular, move pipelock restart work
|
||||||
|
out of direct SIGUSR1 handler execution while preserving the caller-visible
|
||||||
|
`docker kill --signal USR1 <bundle>` contract used by pipelock apply flows.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/sidecar_init.py` is PID 1 for the bundled sidecar container. It
|
||||||
|
starts egress, pipelock, git-gate/git-http, and supervise; forwards shutdown
|
||||||
|
signals; forwards SIGHUP to egress; and restarts pipelock on SIGUSR1 after
|
||||||
|
allowlist changes.
|
||||||
|
|
||||||
|
The current SIGUSR1 handler calls `sup.restart_daemon("pipelock")` directly.
|
||||||
|
That restart path can terminate a child, wait up to a grace timeout, SIGKILL a
|
||||||
|
stubborn child, spawn a replacement with `subprocess.Popen`, and start a new
|
||||||
|
pump thread. In CPython signal handlers run between bytecodes in the main
|
||||||
|
thread, so this is not the same as POSIX async-signal-unsafe C code, but it
|
||||||
|
still means signal handling can block the supervisor loop for the restart grace
|
||||||
|
window and makes stacked signals harder to reason about.
|
||||||
|
|
||||||
|
The exit-code contract is also easy to misread. `_Supervisor.exit_code()`
|
||||||
|
returns the maximum child return code. That preserves a positive crash code
|
||||||
|
when a child failed before graceful shutdown, but the docstring currently
|
||||||
|
frames graceful shutdown as returning zero because signal-killed children have
|
||||||
|
negative return codes. The implementation is reasonable; the contract needs to
|
||||||
|
be deliberate and tested around crash-then-shutdown behavior.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Preserve the external signal contract:
|
||||||
|
- SIGTERM/SIGINT requests bundle shutdown.
|
||||||
|
- SIGHUP still forwards to the live egress child.
|
||||||
|
- SIGUSR1 still requests an in-place pipelock restart.
|
||||||
|
- Keep signal handlers small: handlers should record intent and return, not
|
||||||
|
perform blocking subprocess lifecycle work directly.
|
||||||
|
- Process queued restart requests from the supervisor's main loop so restart
|
||||||
|
behavior is serialized with `tick()` and shutdown state.
|
||||||
|
- Avoid respawning children after shutdown has started.
|
||||||
|
- Coalesce or serialize repeated pipelock restart requests in a documented way
|
||||||
|
so stacked SIGUSR1 delivery cannot overlap restarts.
|
||||||
|
- Clarify and test aggregate exit-code semantics:
|
||||||
|
- clean unattended exits return zero when every child exits zero.
|
||||||
|
- signal-only shutdown does not invent a positive failure code.
|
||||||
|
- a positive child crash before shutdown remains visible on supervisor exit.
|
||||||
|
- Keep the implementation stdlib-only.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No new process supervisor dependency such as supervisord, s6, or runit.
|
||||||
|
- No automatic restart policy for arbitrary unexpected child death.
|
||||||
|
- No changes to the bundle's daemon set, daemon argv, env filtering, or Docker
|
||||||
|
compose contract.
|
||||||
|
- No changes to egress route reload semantics beyond preserving SIGHUP
|
||||||
|
forwarding.
|
||||||
|
- No user-facing CLI changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- Internal signal handling and supervisor event-loop structure in
|
||||||
|
`bot_bottle/sidecar_init.py`.
|
||||||
|
- Tests in `tests/unit/test_sidecar_init.py` for queued restart behavior,
|
||||||
|
shutdown/restart ordering, repeated restart requests, and exit-code
|
||||||
|
semantics.
|
||||||
|
- Docstring/comment updates that describe the concrete signal and exit-code
|
||||||
|
contracts.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Changing pipelock itself to reload config in process.
|
||||||
|
- Restarting egress, git-gate, git-http, or supervise on demand.
|
||||||
|
- Reporting restart events to the supervise MCP plane.
|
||||||
|
- Changing the interim policy that unexpected child death leaves surviving
|
||||||
|
daemons running.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Keep `_Supervisor` as the owner of child process state, but add an explicit
|
||||||
|
pending-action boundary between signal delivery and subprocess lifecycle work.
|
||||||
|
The exact API can be small, for example:
|
||||||
|
|
||||||
|
- `request_shutdown(reason)` keeps its existing idempotent behavior.
|
||||||
|
- `request_restart(daemon_name)` records a pending restart request unless
|
||||||
|
shutdown is already in progress.
|
||||||
|
- `tick()` drains pending restart work before or after child-death logging in a
|
||||||
|
documented order.
|
||||||
|
|
||||||
|
The SIGUSR1 handler should call only the non-blocking request method. The main
|
||||||
|
loop should continue to call `tick()` and sleep on `_POLL_INTERVAL`; `tick()`
|
||||||
|
then performs the actual `restart_daemon("pipelock")` work while normal Python
|
||||||
|
control flow is in the supervisor loop.
|
||||||
|
|
||||||
|
Repeated restart requests should not overlap. Restart requests coalesce by
|
||||||
|
daemon name: if three SIGUSR1 signals arrive before the next loop turn, one
|
||||||
|
pipelock restart is enough because each restart rereads the latest
|
||||||
|
`pipelock.yaml` from disk. This treats SIGUSR1 as "make pipelock reflect the
|
||||||
|
current config" rather than "run exactly one restart per signal."
|
||||||
|
|
||||||
|
Shutdown wins over restart. If SIGTERM/SIGINT is received while a restart is
|
||||||
|
pending, the supervisor should drop the pending restart and terminate live
|
||||||
|
children. If shutdown starts while `restart_daemon` is already executing in the
|
||||||
|
main loop, the existing restart operation may finish, but no additional queued
|
||||||
|
restart should start after shutdown state is set. A simpler implementation may
|
||||||
|
check shutdown only before each queued restart, because signal handlers execute
|
||||||
|
between bytecodes and cannot interrupt a single blocking `wait()` until control
|
||||||
|
returns to Python.
|
||||||
|
|
||||||
|
Exit-code behavior should be documented as "positive failures win, otherwise
|
||||||
|
return zero." Positive process failures remain visible, while a clean shutdown
|
||||||
|
of only zero-exit or signal-terminated children returns zero instead of leaking
|
||||||
|
platform-specific negative signal return codes to the container exit status.
|
||||||
|
|
||||||
|
## Implementation Chunks
|
||||||
|
|
||||||
|
1. Add characterization tests for SIGUSR1 queuing, repeated restart coalescing,
|
||||||
|
shutdown dropping pending restarts, and crash-then-shutdown exit codes.
|
||||||
|
2. Add a pending restart request structure to `_Supervisor` and a
|
||||||
|
non-blocking request method.
|
||||||
|
3. Change the SIGUSR1 handler in `main()` to enqueue the pipelock restart
|
||||||
|
instead of calling `restart_daemon` directly.
|
||||||
|
4. Drain pending restarts from `tick()` with shutdown checks and documented
|
||||||
|
ordering.
|
||||||
|
5. Update docstrings and comments around signal handling and `exit_code()`.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
Run the existing sidecar unit tests:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_sidecar_init`
|
||||||
|
|
||||||
|
Add focused unit tests that avoid process-wide signal handler races where
|
||||||
|
possible by driving `_Supervisor` directly. End-to-end signal tests can remain
|
||||||
|
limited to `main()` behavior that cannot be exercised otherwise.
|
||||||
|
|
||||||
|
Also run the full unit suite before merge:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# PRD 0035: Supervise Wait Bounds
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #128
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Bound the supervise sidecar's request-thread waits so an agent tool call cannot
|
||||||
|
hold an HTTP worker forever while waiting for operator action. Preserve the MCP
|
||||||
|
tool surface, but make timeout behavior explicit, observable, and tested.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/supervise_server.py` handles MCP over a threaded stdlib HTTP
|
||||||
|
server. Tool calls validate a proposal, write it to the supervise queue, and
|
||||||
|
then wait for the operator response file. Today that wait can last forever.
|
||||||
|
Each outstanding tool call consumes one server thread until the operator acts.
|
||||||
|
|
||||||
|
The route-listing helper also performs a live HTTP request to egress inside the
|
||||||
|
request thread. It has a short timeout today, but the behavior is not described
|
||||||
|
as part of a broader request-thread budget.
|
||||||
|
|
||||||
|
This is operationally risky in multi-agent or repeated-call scenarios: a stuck
|
||||||
|
or ignored proposal can accumulate blocked threads, and callers do not get a
|
||||||
|
clear "still pending" answer they can reason about.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Add a bounded wait for operator responses to supervise tool calls.
|
||||||
|
- Return a clear JSON-RPC tool result when the proposal remains pending after
|
||||||
|
the timeout; do not treat pending operator action as an internal server error.
|
||||||
|
- Keep the queued proposal on disk after timeout so the operator can still act.
|
||||||
|
- Make the wait duration configurable by environment with a conservative
|
||||||
|
default.
|
||||||
|
- Preserve current success and rejection result shapes for completed operator
|
||||||
|
responses.
|
||||||
|
- Keep `list-egress-routes` bounded and document its timeout behavior.
|
||||||
|
- Add focused tests for approved responses, timed-out pending responses, and
|
||||||
|
route-list timeout/error handling.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No asynchronous framework or new runtime dependency.
|
||||||
|
- No replacement of the stdlib threaded HTTP server.
|
||||||
|
- No change to the host-side supervise queue format.
|
||||||
|
- No cancellation protocol between the agent and operator UI.
|
||||||
|
- No dashboard or TUI changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/supervise_server.py` request handling.
|
||||||
|
- Any small helper in `bot_bottle/supervise.py` needed to support a bounded
|
||||||
|
wait cleanly.
|
||||||
|
- Unit tests around tool-call response waiting and route-list behavior.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Reworking proposal persistence.
|
||||||
|
- Changing egress apply or pipelock apply flows.
|
||||||
|
- Adding background workers to complete HTTP requests after the client returns.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Introduce a supervise response wait budget,
|
||||||
|
`SUPERVISE_RESPONSE_TIMEOUT_SECONDS`, with a 30 second default. The existing
|
||||||
|
poll loop should stop after that budget and return a normal tool result such as
|
||||||
|
`{"status": "pending", "notes": "operator response timed out; proposal remains queued"}`.
|
||||||
|
The exact field names should fit the existing response schema so agents can
|
||||||
|
handle success, rejection, and pending with one result parser.
|
||||||
|
|
||||||
|
The proposal file must remain in the queue when the HTTP call times out. The
|
||||||
|
operator can still approve or reject it later, but that later response will not
|
||||||
|
resume the original HTTP request.
|
||||||
|
|
||||||
|
Route-listing should continue to use a short HTTP timeout to egress. Errors
|
||||||
|
should be returned as tool results or JSON-RPC errors consistently with the
|
||||||
|
existing server behavior; the implementation should avoid an unbounded socket
|
||||||
|
wait in the request thread.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit-test a tool call whose response appears before the timeout.
|
||||||
|
- Unit-test a tool call whose response never appears and assert the request
|
||||||
|
returns a pending result while the proposal remains queued.
|
||||||
|
- Unit-test invalid timeout env values fall back or fail clearly.
|
||||||
|
- Unit-test `list-egress-routes` timeout/error behavior with a fake URL opener.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_supervise_server`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# PRD 0036: Codex Auth Redaction Policy
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #129
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make Codex host-auth redaction explicit and fixture-driven so dummy
|
||||||
|
`auth.json` generation cannot accidentally preserve future sensitive fields.
|
||||||
|
Keep forwarding only the short-lived host access token through egress, while the
|
||||||
|
guest receives a non-secret auth file whose schema remains useful to Codex.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/codex_auth.py` reads the host Codex auth file, extracts the access
|
||||||
|
token for egress, and writes a dummy guest `auth.json`. The code redacts JWT
|
||||||
|
claims and auth JSON fields with a mix of schema-specific handling and generic
|
||||||
|
placeholder behavior.
|
||||||
|
|
||||||
|
That is safer than copying raw auth, but it is still coverage-sensitive. If
|
||||||
|
Codex adds a new field that carries a token, session identifier, refresh secret,
|
||||||
|
or account metadata and the field name does not match current heuristics, the
|
||||||
|
dummy auth file could preserve more information than intended. Because this is
|
||||||
|
credential-adjacent code, the desired behavior should be allowlist-oriented and
|
||||||
|
backed by explicit fixtures.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Define a durable redaction policy for Codex `auth.json`:
|
||||||
|
- host access token is read for egress only.
|
||||||
|
- guest dummy auth contains no bearer, refresh, session, or secret values.
|
||||||
|
- selected non-secret fields may be preserved only when needed by Codex.
|
||||||
|
- Prefer explicit per-field preservation over broad heuristic pass-through.
|
||||||
|
- Add representative fixture tests for current Codex auth shapes.
|
||||||
|
- Add regression tests for unknown nested fields, sensitive-looking field names,
|
||||||
|
lists, dictionaries, and JWT custom claims.
|
||||||
|
- Preserve dummy token expiration alignment with the host access token.
|
||||||
|
- Keep existing errors for missing, invalid, non-device, or expired auth.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No change to the egress credential-forwarding contract.
|
||||||
|
- No attempt to refresh Codex tokens inside the bottle.
|
||||||
|
- No copying of refresh tokens or raw host auth into the guest.
|
||||||
|
- No dependency on a Codex SDK or external schema package.
|
||||||
|
- No user-facing CLI changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/codex_auth.py` redaction helpers.
|
||||||
|
- Unit tests in `tests/unit/test_codex_auth.py`.
|
||||||
|
- Small documentation comments that distinguish preserved non-secret fields from
|
||||||
|
redacted credential material.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Provider provisioning outside Codex auth file generation.
|
||||||
|
- Egress route construction for Codex.
|
||||||
|
- Runtime calls to Codex/OpenAI services.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Treat the dummy guest `auth.json` as a deliberately synthesized compatibility
|
||||||
|
file, not as a redacted copy of the host file. The implementation may continue
|
||||||
|
to start from the host object for convenience, but preserved fields should be
|
||||||
|
controlled by explicit allowlists at known schema locations.
|
||||||
|
|
||||||
|
At the top level, preserve only `auth_mode`, replace `OPENAI_API_KEY` /
|
||||||
|
`openai_api_key` with `null`, and synthesize the `tokens` block. Unknown scalar
|
||||||
|
top-level fields become placeholders, unknown lists become empty lists, and
|
||||||
|
unknown dictionaries become empty objects.
|
||||||
|
|
||||||
|
In token blocks, replace `access_token` and `id_token` with dummy JWTs, preserve
|
||||||
|
the selected non-secret `account_id`, and redact every other token-block field
|
||||||
|
with the same placeholder / empty container policy. Refresh, session, and future
|
||||||
|
token values are never copied to the guest.
|
||||||
|
|
||||||
|
In JWT payloads, preserve only claims that are known to be non-secret and
|
||||||
|
required for Codex behavior. Unknown scalar claims become placeholders, unknown
|
||||||
|
lists become empty lists, and unknown objects become empty objects.
|
||||||
|
|
||||||
|
For the OpenAI auth claim, preserve only currently necessary non-secret values
|
||||||
|
such as plan type, selected account id, and boolean localhost state. Everything
|
||||||
|
else is placeholder, empty object, or empty list according to the policy.
|
||||||
|
|
||||||
|
Tests should use fixture auth objects that include both current expected fields
|
||||||
|
and intentionally hostile future-looking fields such as `session_context`,
|
||||||
|
`bearer`, `refreshSecret`, nested `token_value`, and opaque arrays. The dummy
|
||||||
|
output must not contain the original secret strings.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Existing `tests/unit/test_codex_auth.py` should continue to pass.
|
||||||
|
- Add tests that assert original access/refresh/session strings do not appear in
|
||||||
|
`codex_dummy_auth_json`.
|
||||||
|
- Add tests for nested JWT and auth-claim redaction behavior.
|
||||||
|
- Add tests that the dummy access/id token `exp` still matches the host access
|
||||||
|
token expiry.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_codex_auth`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# PRD 0037: Pipelock YAML Render Contract
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #130
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Lock down the contract between `pipelock_build_config` and
|
||||||
|
`pipelock_render_yaml` so hand-rendered pipelock YAML stays aligned with the
|
||||||
|
structured config bot-bottle builds. Keep the stdlib-only renderer, but add
|
||||||
|
shape validation and semantic tests for every supported section.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/pipelock.py` builds a structured dict and then renders a fixed YAML
|
||||||
|
shape by hand. This avoids a runtime YAML dependency, but it also means the
|
||||||
|
renderer directly indexes expected keys. If `pipelock_build_config` adds,
|
||||||
|
renames, or conditionalizes a section, rendering can fail at runtime or emit
|
||||||
|
YAML that no longer matches the config semantics.
|
||||||
|
|
||||||
|
Existing tests assert important rendered fragments, but they do not fully lock
|
||||||
|
the build/render contract or optional-section combinations. A mismatch here can
|
||||||
|
weaken DLP enforcement or break bottle launch after a future pipelock policy
|
||||||
|
change.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Keep the renderer stdlib-only.
|
||||||
|
- Define the supported pipelock config shape in one place.
|
||||||
|
- Fail clearly when `pipelock_render_yaml` receives an unsupported or malformed
|
||||||
|
config shape.
|
||||||
|
- Add tests covering all supported sections:
|
||||||
|
- base allowlist and forward proxy.
|
||||||
|
- seed phrase detection toggle.
|
||||||
|
- DLP and request-body/header scanning.
|
||||||
|
- TLS interception and passthrough domains.
|
||||||
|
- SSRF IP allowlist.
|
||||||
|
- Add semantic tests that compare structured config values to rendered YAML
|
||||||
|
output without relying only on brittle substring assertions.
|
||||||
|
- Preserve current rendered YAML for existing configs unless a clearer failure
|
||||||
|
path requires an error message change.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No PyYAML or other runtime dependency.
|
||||||
|
- No change to pipelock policy defaults.
|
||||||
|
- No change to egress-to-pipelock topology.
|
||||||
|
- No change to pipelock image version or config schema beyond validation of the
|
||||||
|
shape bot-bottle already emits.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/pipelock.py` render helpers and validation.
|
||||||
|
- Unit tests in `tests/unit/test_pipelock_yaml.py` and related focused
|
||||||
|
pipelock tests.
|
||||||
|
- Small helper functions for typed access to config sections, if useful.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Launch/backend changes.
|
||||||
|
- Integration tests that start a real pipelock container.
|
||||||
|
- Changing the manifest schema for route-level pipelock policy.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Treat `pipelock_render_yaml` as a serializer for the narrow config shape
|
||||||
|
produced by `pipelock_build_config`, not as a generic YAML renderer. Before
|
||||||
|
rendering a section, validate that required keys exist with the expected
|
||||||
|
primitive/list/dict types. Missing or unsupported shapes should raise a clear
|
||||||
|
`ValueError` naming the section and key.
|
||||||
|
|
||||||
|
The supported top-level shape is `version`, `mode`, `enforce`,
|
||||||
|
`api_allowlist`, `seed_phrase_detection`, `forward_proxy`, `dlp`,
|
||||||
|
`request_body_scanning`, `tls_interception`, and `ssrf`. Required sections are
|
||||||
|
validated before rendering; optional sections keep the current omission
|
||||||
|
behavior. `request_body_scanning.scan_headers`,
|
||||||
|
`request_body_scanning.header_mode`, and
|
||||||
|
`tls_interception.passthrough_domains` remain optional for compatibility with
|
||||||
|
parsed running configs that only contain the older rendered subset.
|
||||||
|
|
||||||
|
Tests should cover both normal output and failure cases. Because the project is
|
||||||
|
stdlib-only, semantic tests can use a small purpose-built parser for the exact
|
||||||
|
rendered shape or compare rendered lines to values from the structured config
|
||||||
|
through helper assertions. The goal is to detect drift between config dict and
|
||||||
|
YAML without adding a general YAML dependency.
|
||||||
|
|
||||||
|
Optional sections should be exercised in combinations:
|
||||||
|
|
||||||
|
- no TLS and no SSRF.
|
||||||
|
- TLS enabled with empty and non-empty passthrough domains.
|
||||||
|
- SSRF enabled with one or more IP/CIDR entries.
|
||||||
|
- all optional sections enabled together.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Extend `tests/unit/test_pipelock_yaml.py` with semantic assertions tying each
|
||||||
|
rendered section back to the config dict.
|
||||||
|
- Add malformed-config tests for missing required keys and wrong section types.
|
||||||
|
- Keep existing render fragment tests where they protect exact pipelock syntax.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_pipelock_yaml`
|
||||||
|
- `python3 -m unittest tests.unit.test_pipelock_allowlist`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# PRD 0038: smolmachines Env Contract and Secret-Safe Injection
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #135
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make smolmachines env handling match Docker's contract: resolve manifest env
|
||||||
|
entries through `resolve_env()`, keep secret and interpolated values out of
|
||||||
|
host argv, and document or enforce an explicit env contract for the backend.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/backend/smolmachines/prepare.py` builds the guest env from
|
||||||
|
`bottle.env` directly, bypassing `resolve_env()`. Entries like `?prompt` and
|
||||||
|
`${HOST_VAR}` can reach the guest literally rather than being prompted or
|
||||||
|
resolved. In contrast, Docker resolves env through `resolve_env()` before
|
||||||
|
writing a mode-600 env file.
|
||||||
|
|
||||||
|
`smolmachines/smolvm.py` renders env as `-e KEY=VALUE` on `smolvm machine
|
||||||
|
create` argv, and `SmolmachinesBottle.agent_argv` / `exec` prepend
|
||||||
|
`env KEY=VALUE …` onto the `smolvm machine exec` argv. Any literal or resolved
|
||||||
|
secret value is therefore visible in the host process table.
|
||||||
|
|
||||||
|
The two backends have no shared env contract document. Divergence will silently
|
||||||
|
widen as new manifest env features are added.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Manifest env entries are resolved through `resolve_env()` before being
|
||||||
|
injected into the smolmachines guest, matching Docker behaviour.
|
||||||
|
- No manifest env value (literal or resolved) appears on host argv during
|
||||||
|
machine creation or exec.
|
||||||
|
- Define and document an explicit smolmachines env contract covering literals,
|
||||||
|
`?prompt` secrets, and `${HOST_VAR}` interpolations.
|
||||||
|
- Unit tests cover: literal passthrough, prompted-secret resolution,
|
||||||
|
host-var interpolation, and the no-argv-leak invariant.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to the Docker env path.
|
||||||
|
- No changes to manifest schema or `resolve_env()` itself.
|
||||||
|
- No changes to smolmachines networking or mount handling.
|
||||||
|
- No new runtime dependencies.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/backend/smolmachines/prepare.py` env resolution.
|
||||||
|
- `bot_bottle/backend/smolmachines/smolvm.py` machine-create argv.
|
||||||
|
- `bot_bottle/backend/smolmachines/bottle.py` `agent_argv` / `exec` env
|
||||||
|
injection.
|
||||||
|
- `bot_bottle/env.py` if helper changes are needed to support the smolmachines
|
||||||
|
path.
|
||||||
|
- Unit tests in `tests/unit/` covering the above.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Integration tests that start a live smolmachines VM.
|
||||||
|
- Docker backend changes.
|
||||||
|
- Dashboard or CLI changes.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Run smolmachines env through `resolve_env()` at prepare time, exactly as Docker
|
||||||
|
does. After resolution, inject env into the guest through a mechanism that does
|
||||||
|
not expose values on host argv — for example by writing a mode-600 env file
|
||||||
|
into the machine's state directory and loading it at exec time, or by passing
|
||||||
|
env through `smolvm`'s stdin if the tool supports it.
|
||||||
|
|
||||||
|
If `smolvm` provides no stdin or env-file injection path, document this as a
|
||||||
|
known limitation and at minimum move env values behind a per-invocation
|
||||||
|
tmpfile rather than inline argv.
|
||||||
|
|
||||||
|
The env contract for smolmachines should mirror Docker's:
|
||||||
|
|
||||||
|
- Literals: passed as-is after resolution.
|
||||||
|
- `?prompt` entries: prompted at prepare time; resolved value injected, never
|
||||||
|
on argv.
|
||||||
|
- `${HOST_VAR}` entries: interpolated from the operator's env at prepare time;
|
||||||
|
resolved value injected, never on argv.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit tests for `prepare.py` asserting `resolve_env()` is called and that
|
||||||
|
resolution results are used rather than raw `bottle.env` values.
|
||||||
|
- Unit tests for `smolvm.py` machine-create argv asserting no env value appears
|
||||||
|
inline.
|
||||||
|
- Unit tests for `bottle.py` exec path asserting the same argv invariant.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_smolmachines_prepare`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- Does `smolvm machine create` support an env-file flag or stdin injection that
|
||||||
|
avoids `-e KEY=VALUE` argv?
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# PRD 0039: smolmachines Capability-Block Remediation
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #136
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make capability-block remediation backend-aware. Today the dashboard approval
|
||||||
|
path calls Docker-only teardown and apply code regardless of which backend
|
||||||
|
created the bottle. Either implement smolmachines remediation or add a clean
|
||||||
|
disable/unsupported path so operators never get a partial Docker teardown
|
||||||
|
against a smolmachines slug.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/cli/dashboard.py` dispatches every capability-block approval to
|
||||||
|
`bot_bottle/backend/docker/capability_apply.py`. That code snapshots with
|
||||||
|
`docker cp`, pushes via `docker exec`, rewrites a Dockerfile override, and
|
||||||
|
removes Docker containers and networks. It does not stop or delete a smolvm
|
||||||
|
machine.
|
||||||
|
|
||||||
|
smolmachines bottles still receive the capability-block supervise tool through
|
||||||
|
`backend/smolmachines/provision/supervise.py`, so agents can queue a
|
||||||
|
remediation the host cannot correctly apply. A partial Docker teardown against
|
||||||
|
a smolmachines slug corrupts neither backend cleanly.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Capability-block approval is routed to backend-specific code.
|
||||||
|
- For the smolmachines backend, either:
|
||||||
|
a. A real remediation implementation that stops the VM, applies the
|
||||||
|
capability change, and restarts correctly; or
|
||||||
|
b. A clean unsupported response that tells the operator the action cannot
|
||||||
|
be taken and leaves the bottle in a consistent state.
|
||||||
|
- If option (b): smolmachines agents do not receive the capability-block tool,
|
||||||
|
so the operator is never prompted for an action that will fail.
|
||||||
|
- Unit tests cover the dispatch logic and the smolmachines path.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to the Docker capability-apply path.
|
||||||
|
- No changes to other supervise tools (cred-block, pipelock-block).
|
||||||
|
- No changes to manifest or egress configuration.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/cli/dashboard.py` approval dispatch.
|
||||||
|
- `bot_bottle/backend/smolmachines/provision/supervise.py` tool registration.
|
||||||
|
- New or updated backend-specific capability apply/disable module for
|
||||||
|
smolmachines.
|
||||||
|
- Unit tests for dispatch routing and smolmachines path.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Changes to `backend/docker/capability_apply.py` internals.
|
||||||
|
- Integration tests that exercise a live smolmachines VM remediation.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Introduce a backend-aware dispatch at the approval call site. Each backend
|
||||||
|
exposes a capability remediation entry point; the dashboard calls the one that
|
||||||
|
matches the bottle's backend. If the backend does not support remediation,
|
||||||
|
the entry point returns a structured error that the dashboard surfaces as an
|
||||||
|
operator message without attempting any teardown.
|
||||||
|
|
||||||
|
If option (b) is chosen initially, suppress capability-block registration in
|
||||||
|
`smolmachines/provision/supervise.py` so agents never see the tool.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit test that approval dispatch selects the smolmachines path for a
|
||||||
|
smolmachines bottle and the Docker path for a Docker bottle.
|
||||||
|
- Unit test for the smolmachines path (unsupported response or real apply).
|
||||||
|
- Regression test that Docker approval still calls `capability_apply.py`.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- Is a real smolmachines capability-apply implementation in scope for this PRD,
|
||||||
|
or should it be deferred to a follow-on after PRD 0040 lands?
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# PRD 0040: Backend-Aware Resume and Dashboard Reattach
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #137
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Persist the backend name in `BottleMetadata` and thread it through `resume` and
|
||||||
|
dashboard reattach so both flows construct the correct backend bottle without
|
||||||
|
relying on env overrides or defaulting to Docker.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`BottleMetadata` records identity, agent, cwd, started_at, and compose project,
|
||||||
|
but not the backend name. Without it:
|
||||||
|
|
||||||
|
- `cli/resume.py` cannot select the right backend from a preserved state dir
|
||||||
|
alone; operators must remember to set `BOT_BOTTLE_BACKEND=smolmachines`
|
||||||
|
separately.
|
||||||
|
- `cli/dashboard.py` `_bottle_for_slug` constructs a `DockerBottle` for any
|
||||||
|
externally discovered slug, so reattaching to a live smolmachines agent
|
||||||
|
from the dashboard sends Docker commands to a smolvm machine.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `BottleMetadata` includes the backend name, written at bottle creation time
|
||||||
|
for both Docker and smolmachines.
|
||||||
|
- `cli resume` reads the persisted backend name and constructs the correct
|
||||||
|
bottle type without requiring an env override.
|
||||||
|
- Dashboard reattach (`_bottle_for_slug`) reads the persisted backend name and
|
||||||
|
constructs the correct bottle type.
|
||||||
|
- Existing Docker bottles without a persisted backend name fall back to Docker
|
||||||
|
(backward-compatible default).
|
||||||
|
- Unit tests cover write, read, backward-compatible fallback, and both
|
||||||
|
resume/reattach code paths.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to manifest or egress configuration.
|
||||||
|
- No new CLI flags (backend selection at resume time should be automatic).
|
||||||
|
- No smolmachines capability-apply implementation (see PRD 0039).
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/backend/docker/bottle_state.py` `BottleMetadata` schema and
|
||||||
|
write path.
|
||||||
|
- `bot_bottle/backend/docker/bottle.py` and
|
||||||
|
`bot_bottle/backend/smolmachines/bottle.py` metadata write at creation.
|
||||||
|
- `bot_bottle/cli/resume.py` backend selection from metadata.
|
||||||
|
- `bot_bottle/cli/dashboard.py` `_bottle_for_slug` backend selection.
|
||||||
|
- Unit tests covering the above.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Migration tooling for existing state dirs.
|
||||||
|
- Integration tests that exercise full resume across process restarts.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Add a `backend` field to `BottleMetadata` with a default of `"docker"` for
|
||||||
|
backward compatibility. Both `DockerBottle` and `SmolmachinesBottle` write
|
||||||
|
their backend name into metadata at creation time.
|
||||||
|
|
||||||
|
`resume` reads the metadata before constructing the bottle object and selects
|
||||||
|
the appropriate backend class. `_bottle_for_slug` does the same. A helper
|
||||||
|
function in the metadata module can encapsulate the backend-name-to-class
|
||||||
|
mapping so the logic is not duplicated.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit tests for `BottleMetadata` serialisation with and without the backend
|
||||||
|
field.
|
||||||
|
- Unit tests for the backward-compatible default.
|
||||||
|
- Unit tests for `resume` selecting smolmachines vs Docker from metadata.
|
||||||
|
- Unit tests for `_bottle_for_slug` selecting smolmachines vs Docker.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# PRD 0041: Git HTTP Request Bounds
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #138
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add Content-Length validation and a body-size cap to `git_http_backend.py` so malformed or oversized smart-HTTP requests fail cleanly rather than crashing the handler or exhausting memory.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
|
||||||
|
|
||||||
|
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- A missing or non-numeric Content-Length returns HTTP 400.
|
||||||
|
- A negative Content-Length returns HTTP 400.
|
||||||
|
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413.
|
||||||
|
- Valid Git smart-HTTP pushes and fetches continue to work.
|
||||||
|
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to git-gate authentication or route logic.
|
||||||
|
- No changes to `supervise_server.py`.
|
||||||
|
- No streaming / chunked-transfer-encoding support.
|
||||||
|
- No TLS changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/git_http_backend.py` request parsing and body reading.
|
||||||
|
- Unit tests in `tests/unit/test_git_http_backend.py`.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Integration tests that drive a real Git client through the handler.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit tests using `unittest.mock` to drive the handler with crafted headers.
|
||||||
|
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_git_http_backend`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
# PRD 0042: smolmachines Cross-Backend Parity Tests
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #139
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add tests that prove secrets, forwarded env, resume, and remediation behave
|
||||||
|
equivalently across Docker and smolmachines backends. The fixes in PRDs
|
||||||
|
0038–0040 are unverifiable without this coverage.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The existing unit suite is broad but backend-specific. There are no tests that
|
||||||
|
run the same scenario against both Docker and smolmachines and assert the
|
||||||
|
outcomes match. A regression in one backend goes undetected until a live run,
|
||||||
|
and PRDs 0038–0040 can each pass their own unit tests while the backends still
|
||||||
|
diverge at the integration boundary.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- A parity test suite that covers at least:
|
||||||
|
- Secret env injection: `?prompt` and `${HOST_VAR}` entries produce the same
|
||||||
|
guest env on both backends.
|
||||||
|
- Forwarded env: literal manifest env values reach the guest on both backends.
|
||||||
|
- Resume: a preserved bottle state dir round-trips correctly on both backends
|
||||||
|
(relies on PRD 0040 metadata).
|
||||||
|
- Remediation: capability-block approval routes to the correct backend handler
|
||||||
|
(relies on PRD 0039 dispatch).
|
||||||
|
- Each scenario is parameterised so a failure names the backend that regressed.
|
||||||
|
- Tests run without a live VM or Docker daemon (mock or stub backends).
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No end-to-end agent execution tests.
|
||||||
|
- No performance or load tests.
|
||||||
|
- No changes to production code (test-only PRD).
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- New test file(s) under `tests/unit/` for parity scenarios.
|
||||||
|
- Stub or mock implementations of smolmachines and Docker backends as needed.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Changes to `bot_bottle/` production code.
|
||||||
|
- CI infrastructure changes beyond adding the new test file to the discover
|
||||||
|
invocation.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- PRD 0038 should land before the env parity tests are finalised.
|
||||||
|
- PRDs 0039 and 0040 should land before the remediation and resume scenarios
|
||||||
|
are finalised; stubs can be written speculatively beforehand.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Parameterise each scenario over a list of backend factory functions. Each
|
||||||
|
factory returns a bottle instance wired to a stub subprocess layer. The test
|
||||||
|
body is backend-agnostic: it calls the same public API, captures the same
|
||||||
|
observable output, and asserts equality.
|
||||||
|
|
||||||
|
For env scenarios, capture the argv or env-file content passed to the guest
|
||||||
|
and compare against resolved manifest values. For resume, write metadata with
|
||||||
|
one backend class and read it back to verify correct selection. For remediation,
|
||||||
|
assert dispatch selects the per-backend handler.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
Run as part of the standard unit discover:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
Or directly:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_backend_parity`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- Should parity tests live under `tests/unit/` (mock-based) or
|
||||||
|
`tests/integration/` (live infra)? Mock-based is preferred to keep CI simple.
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# PRD 0043: Sidecar Pipe Lifecycle Cleanup
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #140
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Close the unclosed child stdout pipe file descriptors that `sidecar_init.py`
|
||||||
|
leaks during restart and shutdown paths, eliminating `ResourceWarning` noise
|
||||||
|
and tightening the process lifecycle.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Unit tests for `sidecar_init.py` pass, but restart and shutdown cases emit
|
||||||
|
`ResourceWarning: unclosed file <_io.BufferedReader …>` for child stdout pipes,
|
||||||
|
originating around lines 141 and 273. The warnings indicate the restart path
|
||||||
|
leaks pipe file descriptors: a pipe opened for a stopped or replaced child is
|
||||||
|
not explicitly closed before the next child is spawned or before the supervisor
|
||||||
|
exits.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_sidecar_init` produces no
|
||||||
|
`ResourceWarning` output.
|
||||||
|
- Pipe file descriptors for stopped or replaced child processes are explicitly
|
||||||
|
closed in the restart path.
|
||||||
|
- Pipe file descriptors for all children are explicitly closed in the shutdown
|
||||||
|
path.
|
||||||
|
- No change to the external signal or exit-code contract from PRD 0034.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to restart or shutdown policy (coalescing, ordering, timeout).
|
||||||
|
- No changes to egress, pipelock, git-gate, or supervise daemon argv.
|
||||||
|
- No new runtime dependencies.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/sidecar_init.py` pipe open/close lifecycle in `_Supervisor`.
|
||||||
|
- Unit tests in `tests/unit/test_sidecar_init.py` asserting no leaked pipes.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Changing how pumping threads read from pipes.
|
||||||
|
- Integration tests that start a live sidecar container.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Audit every code path in `_Supervisor` where a child process is stopped,
|
||||||
|
replaced, or reaches end-of-life, and ensure the corresponding stdout pipe is
|
||||||
|
explicitly closed before spawning a replacement or exiting the supervisor loop.
|
||||||
|
|
||||||
|
Where a pumping thread holds a reference to the pipe, coordinate closure so the
|
||||||
|
thread sees EOF and exits cleanly rather than blocking indefinitely.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Enable `ResourceWarning` as an error in test setUp:
|
||||||
|
`warnings.simplefilter("error", ResourceWarning)`.
|
||||||
|
- Run existing restart and shutdown test cases under this stricter setting.
|
||||||
|
- Add tests for restart-then-shutdown if not already covered.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest tests.unit.test_sidecar_init`
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
# PRD 0044: Print Parity Across Backends
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #96
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Hoist `git_gate_plan`, `egress_plan`, `agent_provision`, and `supervise_plan`
|
||||||
|
from the concrete `BottlePlan` subclasses up to `BottlePlan`, and implement
|
||||||
|
`print` concretely there. This eliminates the two per-backend output divergences
|
||||||
|
and ensures any future backend gets correct preflight rendering for free.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`BottlePlan.print` is `@abstractmethod`, so each backend provides its own
|
||||||
|
implementation. The two current implementations have drifted:
|
||||||
|
|
||||||
|
| Field | Docker | smolmachines |
|
||||||
|
|---|---|---|
|
||||||
|
| git gate lines | `upstream_host:upstream_port` from resolved `git_gate_plan.upstreams` | `Name → Upstream` from manifest `bottle.git` |
|
||||||
|
| egress lines | `host [auth:scheme]` | `host` only (auth dropped) |
|
||||||
|
|
||||||
|
The smolmachines docstring says "same shape as the Docker backend's so operators
|
||||||
|
see one format across backends" — that intent is real but nothing enforces it.
|
||||||
|
|
||||||
|
The env_names divergence previously noted in this issue was resolved by PRD 0038
|
||||||
|
(smolmachines env contract): `resolved.forwarded` is now merged into
|
||||||
|
`agent_provision.guest_env` at prepare time on both backends, so displayed env
|
||||||
|
names are equivalent.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `BottlePlan` carries `git_gate_plan`, `egress_plan`, `agent_provision`, and
|
||||||
|
`supervise_plan` as concrete fields; subclasses no longer declare them
|
||||||
|
independently.
|
||||||
|
- `BottlePlan.print` is a concrete method; subclasses have no `print`
|
||||||
|
implementation of their own.
|
||||||
|
- Both backends render git gate lines as `name → upstream_host:upstream_port`
|
||||||
|
(using `git_gate_plan.upstreams`), not the manifest-level URL.
|
||||||
|
- Both backends render egress lines as `host [auth:scheme]` (dropping the
|
||||||
|
annotation only when `auth_scheme` is empty).
|
||||||
|
- Unit tests assert the unified output for both backends from a single shared
|
||||||
|
test helper.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No changes to the Docker or smolmachines launch, prepare, or cleanup paths.
|
||||||
|
- No changes to how env values are resolved or injected (that is PRD 0038).
|
||||||
|
- No changes to the manifest schema or `GitEntry`.
|
||||||
|
- No new CLI flags or dashboard changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `bot_bottle/backend/__init__.py` — add `git_gate_plan`, `egress_plan`,
|
||||||
|
`agent_provision`, and `supervise_plan` fields to `BottlePlan`; replace
|
||||||
|
`@abstractmethod print` with a concrete implementation.
|
||||||
|
- `bot_bottle/backend/docker/bottle_plan.py` — remove the four hoisted fields
|
||||||
|
and the `print` method.
|
||||||
|
- `bot_bottle/backend/smolmachines/bottle_plan.py` — remove the four hoisted
|
||||||
|
fields and the `print` method.
|
||||||
|
- `tests/unit/` — add or update tests asserting unified preflight output; a
|
||||||
|
shared helper can build a minimal plan fixture for each backend and assert
|
||||||
|
the same lines appear.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Changes to `bot_bottle/backend/print_util.py` beyond what the new `print`
|
||||||
|
implementation requires.
|
||||||
|
- Changes to `BottleCleanupPlan.print` or any other print method.
|
||||||
|
- Integration tests that launch a real bottle.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Move the four fields that both concrete subclasses already declare —
|
||||||
|
`git_gate_plan: GitGatePlan`, `egress_plan: EgressPlan`,
|
||||||
|
`agent_provision: AgentProvisionPlan`, `supervise_plan: SupervisePlan | None`
|
||||||
|
— up to `BottlePlan`. Both backends' `prepare` paths already produce these with
|
||||||
|
the same types, so no prepare-time changes are needed.
|
||||||
|
|
||||||
|
Replace the `@abstractmethod` `print` with a concrete implementation on
|
||||||
|
`BottlePlan` that:
|
||||||
|
|
||||||
|
1. Builds `env_names` from `bottle.env.keys() | agent_provision.guest_env.keys()`
|
||||||
|
filtered through `agent_provision.hidden_env_names`.
|
||||||
|
2. Builds git gate lines from `git_gate_plan.upstreams` as
|
||||||
|
`f"{u.name} → {u.upstream_host}:{u.upstream_port}"`.
|
||||||
|
3. Builds egress lines from `egress_plan.routes` as
|
||||||
|
`f"{r.host} [auth:{r.auth_scheme}]"` when `r.auth_scheme` is non-empty,
|
||||||
|
else `r.host`.
|
||||||
|
4. Renders the standard two-column preflight block (leading blank line, agent,
|
||||||
|
provider, env, skills, bottle, git identity, git gate, egress, trailing blank
|
||||||
|
line).
|
||||||
|
|
||||||
|
Docker's `forwarded_env` keys are already merged into `agent_provision.guest_env`
|
||||||
|
via the `agent_provision_plan` builder, so no special handling is needed for
|
||||||
|
env_names.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Add a shared fixture builder (e.g. `make_plan(backend)`) in a new or existing
|
||||||
|
unit test module that constructs a minimal `DockerBottlePlan` and
|
||||||
|
`SmolmachinesBottlePlan` from the same spec and plan fields.
|
||||||
|
- Assert that `plan.print(remote_control=False)` produces identical git gate and
|
||||||
|
egress lines for both backends given the same `git_gate_plan` and
|
||||||
|
`egress_plan`.
|
||||||
|
- Test the `auth_scheme` annotation: present when non-empty, absent otherwise.
|
||||||
|
- Test git gate rendering: `name → host:port` format.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
# PRD 0045: Workspace Porting Plan
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #116
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Add a backend-neutral `WorkspacePlan` that describes how the operator's current
|
||||||
|
workspace is represented inside a bottle. Docker and smolmachines should both
|
||||||
|
use this plan for workspace path, working directory, content copy, `.git` copy,
|
||||||
|
ownership, and provider trust configuration instead of rediscovering
|
||||||
|
`/home/node/workspace` in separate launch and provisioning code paths.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The current `--cwd` behavior is spread across backend-specific code:
|
||||||
|
|
||||||
|
- Docker builds a derived image that copies the host cwd to
|
||||||
|
`/home/node/workspace`, sets that as `WORKDIR`, and patches Claude trust in
|
||||||
|
the generated Dockerfile.
|
||||||
|
- Docker git provisioning separately copies `.git` into
|
||||||
|
`/home/node/workspace/.git`.
|
||||||
|
- smolmachines git provisioning reconstructs `<guest_home>/workspace/.git`, but
|
||||||
|
does not copy the full working tree.
|
||||||
|
- Codex provider setup trusts `guest_home`, not the copied workspace path.
|
||||||
|
|
||||||
|
These details create backend drift and make provider-specific workspace fixes
|
||||||
|
easy to hard-code in the wrong layer.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `BottleSpec` remains the CLI intent shape (`copy_cwd`, `user_cwd`), while a
|
||||||
|
resolved `WorkspacePlan` carries the backend-neutral guest workspace contract.
|
||||||
|
- `BottlePlan` exposes `workspace_plan` so shared and backend-specific
|
||||||
|
provisioning paths consume one resolved object.
|
||||||
|
- The default in-bottle workspace path remains `/home/node/workspace` when
|
||||||
|
`--cwd` is enabled.
|
||||||
|
- Docker uses `WorkspacePlan` when building the derived cwd image and when
|
||||||
|
provisioning cwd `.git` state.
|
||||||
|
- smolmachines copies the host cwd contents into the same logical workspace
|
||||||
|
path and uses `WorkspacePlan` when provisioning cwd `.git` state.
|
||||||
|
- Provider trust configuration is written for the workspace path when `--cwd`
|
||||||
|
is enabled, and for the guest home when `--cwd` is disabled.
|
||||||
|
- Unit tests cover plan resolution, provider trust path selection, Docker
|
||||||
|
derived image rendering, and both backends' `.git` copy targets.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No new user-facing flags for custom workspace paths.
|
||||||
|
- No manifest schema changes.
|
||||||
|
- No redesign of git-gate or `bottle.git` entries.
|
||||||
|
- No switch from Docker image-copy to bind-mount.
|
||||||
|
- No unrelated provider auth changes.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- Add a small workspace planning module.
|
||||||
|
- Add `workspace_plan` to `BottlePlan` and populate it in Docker and
|
||||||
|
smolmachines prepare paths.
|
||||||
|
- Thread the trusted project path into provider provisioning.
|
||||||
|
- Replace hard-coded `/home/node/workspace` cwd copy and `.git` copy sites with
|
||||||
|
`WorkspacePlan` values.
|
||||||
|
- Copy full host cwd contents for smolmachines `--cwd` parity.
|
||||||
|
- Update focused unit tests.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Integration tests that launch real Docker containers or smolmachines VMs.
|
||||||
|
- Path customization in the bottle manifest or CLI.
|
||||||
|
- Runtime synchronization after bottle launch; this remains a launch-time copy.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Add `bot_bottle/workspace.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WorkspacePlan:
|
||||||
|
enabled: bool
|
||||||
|
host_path: Path
|
||||||
|
guest_home: str
|
||||||
|
guest_path: str
|
||||||
|
workdir: str
|
||||||
|
owner: str = "node:node"
|
||||||
|
mode: str = "755"
|
||||||
|
copy_contents: bool = True
|
||||||
|
copy_git: bool = True
|
||||||
|
has_host_git_dir: bool = False
|
||||||
|
```
|
||||||
|
|
||||||
|
`workspace_plan(spec, guest_home)` resolves:
|
||||||
|
|
||||||
|
- `enabled` from `spec.copy_cwd`.
|
||||||
|
- `host_path` from `spec.user_cwd`.
|
||||||
|
- `guest_path` as `<guest_home>/workspace` when enabled, else `guest_home`.
|
||||||
|
- `workdir` as `guest_path` when enabled, else `guest_home`.
|
||||||
|
- `has_host_git_dir` from `<host_path>/.git`.
|
||||||
|
|
||||||
|
Backends resolve this in `prepare` using their existing guest-home knobs:
|
||||||
|
|
||||||
|
- Docker: `BOT_BOTTLE_CONTAINER_HOME`, default `/home/node`.
|
||||||
|
- smolmachines: `BOT_BOTTLE_GUEST_HOME`, default `/home/node`.
|
||||||
|
|
||||||
|
`BottlePlan` carries the result so launch, git provisioning, and provider
|
||||||
|
provisioning stop consulting `spec.copy_cwd` and hard-coded paths directly.
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
|
||||||
|
Keep the current derived-image transport. Change
|
||||||
|
`build_image_with_cwd(derived, base, cwd)` to accept a `WorkspacePlan` or
|
||||||
|
explicit guest path/workdir fields, then render:
|
||||||
|
|
||||||
|
- `COPY --chown=node:node . <workspace_plan.guest_path>`
|
||||||
|
- `WORKDIR <workspace_plan.workdir>`
|
||||||
|
|
||||||
|
Claude trust should move out of the generated cwd Dockerfile and into provider
|
||||||
|
provisioning so Docker and smolmachines share the same provider trust behavior.
|
||||||
|
|
||||||
|
### smolmachines
|
||||||
|
|
||||||
|
Copy host cwd contents into `workspace_plan.guest_path` during provisioning or
|
||||||
|
VM initialization, then chown the resulting workspace to `node:node`. Continue
|
||||||
|
to copy `.git` through the existing smolvm transport, but target
|
||||||
|
`<workspace_plan.guest_path>/.git`.
|
||||||
|
|
||||||
|
This intentionally closes the current parity gap where smolmachines receives
|
||||||
|
repo metadata without the working tree.
|
||||||
|
|
||||||
|
### Provider Trust
|
||||||
|
|
||||||
|
Extend provider planning with a `trusted_project_path` argument. Callers pass
|
||||||
|
`workspace_plan.workdir`.
|
||||||
|
|
||||||
|
Codex writes:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[projects."<trusted_project_path>"]
|
||||||
|
trust_level = "trusted"
|
||||||
|
```
|
||||||
|
|
||||||
|
Claude writes or updates `.claude.json` so `projects` includes
|
||||||
|
`trusted_project_path` with `hasTrustDialogAccepted: true`. This provisioning
|
||||||
|
belongs in `AgentProvisionPlan` so both backends apply it through their existing
|
||||||
|
provider file-copy primitives.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- Unit-test `workspace_plan()` for enabled and disabled cwd, guest-home
|
||||||
|
overrides, and `.git` detection.
|
||||||
|
- Unit-test Docker cwd image rendering to prove it uses the plan's guest path
|
||||||
|
and workdir.
|
||||||
|
- Unit-test provider planning for Codex and Claude trusted project paths.
|
||||||
|
- Unit-test Docker and smolmachines git provisioning targets using mocked copy
|
||||||
|
and exec primitives.
|
||||||
|
- Unit-test smolmachines workspace content copy target and ownership command.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# PRD 0046: Remove Git Remote Host Overrides
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-codex
|
||||||
|
- **Created:** 2026-06-02
|
||||||
|
- **Issue:** #152
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Remove git remote host override plumbing from bottle manifests and git-gate
|
||||||
|
startup. Git remote declarations should describe upstream repositories and the
|
||||||
|
git-gate credential material needed to mirror them; they should not also
|
||||||
|
configure hosts-file behavior for sidecars.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The git remote model currently has a hosts override path that can make a git
|
||||||
|
upstream resolve differently inside the git-gate sidecar. That is surprising
|
||||||
|
because the same hostname may also be used for HTTP/API traffic that should keep
|
||||||
|
using the normal egress DNS and policy path.
|
||||||
|
|
||||||
|
Keeping host resolution in the git remote model makes repository routing,
|
||||||
|
sidecar hosts files, and egress behavior feel coupled even when the operator
|
||||||
|
only meant to configure git-gate.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- Git remote manifest parsing no longer stores host override data.
|
||||||
|
- Git-gate upstream plans no longer carry host override data.
|
||||||
|
- Docker compose rendering no longer emits sidecar `extra_hosts` entries from
|
||||||
|
git remote declarations.
|
||||||
|
- Smolmachines bundle launch planning has no unused host override path for
|
||||||
|
git-gate.
|
||||||
|
- Focused unit tests cover the absence of sidecar `extra_hosts` for git
|
||||||
|
upstreams.
|
||||||
|
- Current user-facing documentation no longer advertises git remote host
|
||||||
|
overrides.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No replacement hosts-file override feature.
|
||||||
|
- No SSH client config provisioning.
|
||||||
|
- No change to git-gate's SSH credential or known-host handling.
|
||||||
|
- No change to egress DNS, HTTP auth, or pipelock routing semantics.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Remove the host override field from the internal `GitEntry` and
|
||||||
|
`GitGateUpstream` models. Remove the git-gate aggregation helper and the Docker
|
||||||
|
compose code that converted those values into sidecar `extra_hosts`.
|
||||||
|
|
||||||
|
The manifest parser does not need a migration-specific error path. After this
|
||||||
|
change, the old hosts override key has no internal model field and no runtime
|
||||||
|
effect.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
- `python3 -m unittest discover -s tests/unit`
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,170 @@
|
|||||||
|
# PRD 0047: Git-gate Manifest Redesign
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis
|
||||||
|
- **Created:** 2026-06-03
|
||||||
|
- **Issue:** #160
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Replace the `git` top-level key in bottle and agent manifests with `git-gate`,
|
||||||
|
consolidating git-identity configuration (`user`) and git-gate sidecar
|
||||||
|
configuration (`repos`) under a single section. Within `repos`, field names
|
||||||
|
move to lowercase snake_case and the local repo name is promoted to the YAML
|
||||||
|
key. The change removes the ambiguity in the current `git` block: its fields
|
||||||
|
are not generic git or SSH config — they are specifically the credential,
|
||||||
|
host-trust, and identity material that is managed in relation to git-gate.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The current bottle manifest uses a `git` top-level key that mixes two concerns:
|
||||||
|
|
||||||
|
- `git.user` — `git config --global user.name / user.email` identity, which
|
||||||
|
the provisioner injects into the agent's shell.
|
||||||
|
- `git.remotes` — upstream URL, identity file, and host key material that the
|
||||||
|
git-gate sidecar consumes; the agent never sees these values.
|
||||||
|
|
||||||
|
That grouping suggests the `remotes` entries behave like an SSH config or a
|
||||||
|
generic `.gitconfig` remote declaration. They do not. The gate reads the
|
||||||
|
credential material to push upstream after gitleaks passes; the agent's
|
||||||
|
`.gitconfig` receives only the `insteadOf` rewrite that redirects traffic
|
||||||
|
through the gate. Nothing in the current key name or field names signals this.
|
||||||
|
|
||||||
|
Splitting `git.user` into a separate section from `git.remotes` also doesn't
|
||||||
|
help: both concepts exist because of git-gate, and keeping them under a single
|
||||||
|
`git-gate` key makes their relationship and purpose explicit.
|
||||||
|
|
||||||
|
The field names inside each remote entry also use PascalCase (`Name`,
|
||||||
|
`Upstream`, `IdentityFile`, `KnownHostKey`), inconsistent with every other
|
||||||
|
manifest section, which uses snake_case.
|
||||||
|
|
||||||
|
The current `git.remotes` dict is keyed by upstream host, which works for
|
||||||
|
simple remotes but forces a separate `Name` field to give the gate's bare repo
|
||||||
|
a local label. The host key and `Name` field are often redundant or confusing
|
||||||
|
(e.g., IP-literal upstreams where the key carries no semantic meaning).
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `git-gate` is accepted as a top-level bottle and agent key; `git` is removed
|
||||||
|
from both allowed-key sets.
|
||||||
|
- `git-gate.repos` is a named map where each key is the local repo name
|
||||||
|
exposed by the gate (bottle-only; rejected at the agent level).
|
||||||
|
- Each entry in `git-gate.repos` accepts exactly: `url` (required), `identity`
|
||||||
|
(required), `host_key` (optional).
|
||||||
|
- `git-gate.user` replaces `git.user` on both bottles and agents, with the
|
||||||
|
same `name` / `email` fields and overlay semantics.
|
||||||
|
- The manifest parser rejects `git.remotes` and `git.user` with errors that
|
||||||
|
point to the new keys.
|
||||||
|
- `GitEntry` internal fields are updated to match the new names; all callers
|
||||||
|
(provisioner, git-gate render, plan, tests) compile and pass.
|
||||||
|
- Existing unit tests in `tests/unit/test_manifest_git.py` and
|
||||||
|
`tests/unit/test_manifest_git_user.py` are rewritten to use the new YAML
|
||||||
|
shape; all other manifest unit tests remain green.
|
||||||
|
- The demo manifest (`bot-bottle.demo.json`) and any examples using the old
|
||||||
|
shape are updated.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No change to `git.user` / `git-gate.user` semantics or field names (`name`,
|
||||||
|
`email`).
|
||||||
|
- No change to git-gate runtime behavior (mirroring, gitleaks, access-hook
|
||||||
|
refresh).
|
||||||
|
- No change to the `insteadOf` rewrite the provisioner emits.
|
||||||
|
- No migration shim: the old `git.*` shape is rejected immediately with clear
|
||||||
|
error messages pointing to the new keys.
|
||||||
|
- No change to how agent-level user config overlays the bottle-level value.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### New manifest shape
|
||||||
|
|
||||||
|
**Before** (bottle frontmatter):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
git:
|
||||||
|
user:
|
||||||
|
name: implementer-bot
|
||||||
|
email: eric+implementer@dideric.is
|
||||||
|
remotes:
|
||||||
|
gitea.dideric.is:
|
||||||
|
Name: bot-bottle
|
||||||
|
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
|
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
||||||
|
KnownHostKey: "ssh-rsa AAAA..."
|
||||||
|
```
|
||||||
|
|
||||||
|
**After**:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
git-gate:
|
||||||
|
user:
|
||||||
|
name: implementer-bot
|
||||||
|
email: eric+implementer@dideric.is
|
||||||
|
repos:
|
||||||
|
bot-bottle:
|
||||||
|
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
|
identity: ~/.ssh/gitea-delos-2.pem
|
||||||
|
host_key: "ssh-rsa AAAA..."
|
||||||
|
```
|
||||||
|
|
||||||
|
`git-gate` is the single optional top-level key for all git configuration.
|
||||||
|
Bottles that previously used only `git.user` now use only `git-gate.user`;
|
||||||
|
those that used only `git.remotes` now use only `git-gate.repos`.
|
||||||
|
|
||||||
|
### Key-name-as-repo-name
|
||||||
|
|
||||||
|
The YAML key in `git-gate.repos` becomes the local repo name (previously
|
||||||
|
`Name`). The upstream host is no longer the primary key; the provisioner and
|
||||||
|
gate derive it from the `url` field during parse. IP-literal upstreams work
|
||||||
|
without an artificial host-as-key constraint.
|
||||||
|
|
||||||
|
### Field renames
|
||||||
|
|
||||||
|
| Old field | New field |
|
||||||
|
|-----------|-----------|
|
||||||
|
| `Name` (from dict key) | YAML key in `repos` |
|
||||||
|
| `Upstream` | `url` |
|
||||||
|
| `IdentityFile` | `identity` |
|
||||||
|
| `KnownHostKey` | `host_key` |
|
||||||
|
|
||||||
|
### Parser changes
|
||||||
|
|
||||||
|
- `manifest_schema.py`: replace `"git"` with `"git-gate"` in `BOTTLE_KEYS`
|
||||||
|
and `AGENT_KEYS_OPTIONAL`.
|
||||||
|
- `manifest.py`: replace `_parse_git_config` with `_parse_git_gate_config`
|
||||||
|
that validates both `user` and `repos` subkeys. Update `Bottle.from_dict`
|
||||||
|
and `Agent.from_dict` to call it for the `"git-gate"` key.
|
||||||
|
- `Agent.from_dict` continues to reject `repos` at the agent level with a
|
||||||
|
clear error.
|
||||||
|
- Remove `from_remote_dict` and update `GitEntry._from_object` to accept the
|
||||||
|
new field names. Internal dataclass field names (`UpstreamUser`, etc.) are
|
||||||
|
unchanged — they are internal plumbing, not user-facing.
|
||||||
|
- Any existing `"git"` key raises a targeted error:
|
||||||
|
|
||||||
|
```
|
||||||
|
bottle 'dev' uses 'git' which has been replaced by 'git-gate' (PRD 0047).
|
||||||
|
Move git.user → git-gate.user and git.remotes → git-gate.repos.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```
|
||||||
|
python3 -m unittest discover -s tests/unit
|
||||||
|
```
|
||||||
|
|
||||||
|
Test files to update:
|
||||||
|
|
||||||
|
- `tests/unit/test_manifest_git.py` — rewrite fixtures and assertions to use
|
||||||
|
`git-gate.repos` / lowercase fields. Cover: minimal entry, optional
|
||||||
|
`host_key`, missing `url`, missing `identity`, unknown key, IP-literal
|
||||||
|
upstreams, duplicate name rejection, old `git.remotes` and bare `git` key
|
||||||
|
both rejected.
|
||||||
|
- `tests/unit/test_manifest_git_user.py` and
|
||||||
|
`tests/unit/test_manifest_agent_git_user.py` — update fixtures to use
|
||||||
|
`git-gate.user` at both bottle and agent level.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Product requirement docs
|
||||||
|
|
||||||
|
One PRD per feature: what to build, why, and how it's scoped. The PRD
|
||||||
|
is the durable spec — it should stand on its own without a Gitea issue
|
||||||
|
thread (see [`../README.md`](../README.md) for when a PRD is the right
|
||||||
|
document vs. a research note or a decision record).
|
||||||
|
|
||||||
|
## Naming and numbering
|
||||||
|
|
||||||
|
`NNNN-kebab-title.md`, zero-padded and sequential (`0024-…`, `0025-…`).
|
||||||
|
Numbers are never reused; gaps are fine (there is no 0005). The number
|
||||||
|
is assigned at creation and stays fixed for the life of the doc.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
The `Status:` line near the top tracks the PRD's lifecycle:
|
||||||
|
|
||||||
|
- **Draft** — proposed, not yet shipped.
|
||||||
|
- **Active** — the design has shipped to `main` and is in effect.
|
||||||
|
- **Superseded by [PRD NNNN](…)** — replaced by a later PRD; kept for history.
|
||||||
|
- **Retargeted by [PRD NNNN](…)** — folded into a later PRD's scope.
|
||||||
|
|
||||||
|
## Format
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# PRD NNNN: <short title>
|
||||||
|
|
||||||
|
- **Status:** Draft
|
||||||
|
- **Author:** <who>
|
||||||
|
- **Created:** YYYY-MM-DD
|
||||||
|
- **Issue:** #<n> # optional — convenience pointer only
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
One paragraph: what this builds and the pain it solves.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
The current state and why it's inadequate.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
Bullets a reviewer can check the finished work against.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
What this explicitly does not do — and won't, to head off scope creep.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
In scope / out of scope, when the boundary needs spelling out.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
How it works: schema, data flow, diagrams, algorithms as needed.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
Ordered, mergeable steps (optional; for multi-PR features).
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
Unresolved decisions — resolve or fold into Design before shipping.
|
||||||
|
```
|
||||||
|
|
||||||
|
Sections are a guide, not a straitjacket: drop the ones a given PRD
|
||||||
|
doesn't need (a small change rarely needs Scope or Implementation
|
||||||
|
chunks) and add others where they help (e.g. Testing strategy,
|
||||||
|
Alternatives considered, References). Keep the rationale self-contained
|
||||||
|
— inline the reasoning rather than linking out to an issue thread, so
|
||||||
|
the PRD survives a move off Gitea.
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Research notes
|
||||||
|
|
||||||
|
Investigations into a question or a design space — landscape surveys,
|
||||||
|
tradeoff analyses, "should we do X or Y," assessments of an approach
|
||||||
|
before (or instead of) committing it to a PRD. A research note is where
|
||||||
|
the *thinking* lives; a PRD is where a decided feature lives, and a
|
||||||
|
decision record is where a settled choice lives (see
|
||||||
|
[`../README.md`](../README.md) for picking between them).
|
||||||
|
|
||||||
|
Notes are opinionated. They reach a conclusion rather than dumping a
|
||||||
|
neutral survey — the point is to move a decision forward and leave a
|
||||||
|
durable record of why it went the way it did.
|
||||||
|
|
||||||
|
## Naming
|
||||||
|
|
||||||
|
`kebab-case-topic.md`, named by subject and **not** numbered (unlike
|
||||||
|
PRDs and decision records). Pick a name that says what was
|
||||||
|
investigated: `bash-vs-python-vs-go.md`, `pipelock-assessment.md`,
|
||||||
|
`issue-tracking-vs-in-repo-decision-history.md`.
|
||||||
|
|
||||||
|
## Shape (freeform)
|
||||||
|
|
||||||
|
There's no fixed template — use whatever structure fits the question.
|
||||||
|
In practice most notes share a loose shape:
|
||||||
|
|
||||||
|
- **Open with the question** — a sentence or two on what's being
|
||||||
|
investigated and why it came up.
|
||||||
|
- **Lead with the verdict** — a `## Summary` near the top stating the
|
||||||
|
conclusion, so a reader gets the answer without reading the whole
|
||||||
|
thing.
|
||||||
|
- **Then the analysis** — whatever the argument needs: comparison
|
||||||
|
tables, per-option sections, failure-mode walkthroughs, the axes that
|
||||||
|
actually matter.
|
||||||
|
- **End with a recommendation** when the note exists to drive a
|
||||||
|
decision.
|
||||||
|
|
||||||
|
Keep the reasoning self-contained and grounded: cite sources, link
|
||||||
|
files and PRDs, and prefer concrete evidence from this repo over
|
||||||
|
generic claims — a note should stand on its own without a chat log or a
|
||||||
|
Gitea thread. When a note's recommendation gets acted on, capture the
|
||||||
|
resulting decision in a PRD or a decision record; the note stays as the
|
||||||
|
"why we looked into it," not the system of record for the choice.
|
||||||
@@ -314,9 +314,9 @@ In priority order:
|
|||||||
npm even if it captures something. Also disable Sentry error
|
npm even if it captures something. Also disable Sentry error
|
||||||
reporting via `DISABLE_ERROR_REPORTING=1`.
|
reporting via `DISABLE_ERROR_REPORTING=1`.
|
||||||
|
|
||||||
3. **Generalize the same proxy to forge tokens.** Add a manifest
|
3. **Generalize the same proxy to Git-host tokens.** Add a manifest
|
||||||
field along the lines of
|
field along the lines of
|
||||||
`forge: { kind: "gitea", url, tokenRef }` so a per-bottle token
|
`git_host: { kind: "gitea", url, tokenRef }` so a per-bottle token
|
||||||
reference resolves at launch, the proxy starts as root before
|
reference resolves at launch, the proxy starts as root before
|
||||||
`node` is exec'd, and `tea` plus git HTTPS remotes are
|
`node` is exec'd, and `tea` plus git HTTPS remotes are
|
||||||
pre-configured to point at the proxy. Use
|
pre-configured to point at the proxy. Use
|
||||||
|
|||||||
@@ -0,0 +1,196 @@
|
|||||||
|
# Tracking feature requests in Gitea vs. in-repo decision history
|
||||||
|
|
||||||
|
Research into whether bot-bottle should track feature requests (and the
|
||||||
|
decision-making around them) as Gitea issues, given that the project
|
||||||
|
already records specs in-repo as PRDs (`docs/prds/`) and rationale as
|
||||||
|
research notes (`docs/research/`). The stated constraint is that the
|
||||||
|
*history of why we decided things* should be durable and portable —
|
||||||
|
not locked into a single hosting provider (Gitea today, conceivably
|
||||||
|
GitHub or something else tomorrow).
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Keep using issues, but demote them. The repository — not Gitea — is
|
||||||
|
the system of record for any decision you would be unhappy to lose.
|
||||||
|
Issues are an excellent **inbox and coordination surface** (cheap
|
||||||
|
capture, triage, async discussion, notifications, auto-linking) and a
|
||||||
|
**poor archive** (provider-locked storage, brittle numeric references,
|
||||||
|
rationale stranded in comment threads). The failure mode to avoid is the
|
||||||
|
one already present in the repo: a PRD whose reasoning is only complete
|
||||||
|
if you also read a Gitea issue thread.
|
||||||
|
|
||||||
|
The fix is a discipline, not a tool: **every load-bearing decision gets
|
||||||
|
reified into a versioned file in the repo before the issue that prompted
|
||||||
|
it is closed.** PRDs already do this for features; the gap is (a) small
|
||||||
|
requests that never merit a PRD and (b) decisions that aren't features
|
||||||
|
at all (e.g. "we merge with rebase," "author identity is claimed-not-
|
||||||
|
vouched"). Close that gap with a lightweight in-repo decision log. Then
|
||||||
|
issues can be as disposable as Gitea makes them, and migrating off
|
||||||
|
Gitea costs you triage state, not history.
|
||||||
|
|
||||||
|
## Why this even comes up here
|
||||||
|
|
||||||
|
The project already leans on the repo for durable artifacts:
|
||||||
|
|
||||||
|
- **PRDs** (`docs/prds/0001…0027`) — the spec and its rationale.
|
||||||
|
- **Research notes** (`docs/research/`) — the "why," with tradeoffs.
|
||||||
|
- **Conventional-commit history** — a machine-greppable change log.
|
||||||
|
|
||||||
|
But the issue layer has quietly become load-bearing in places:
|
||||||
|
|
||||||
|
- PRD 0025 says it picked "option 3" *"from the #88 design
|
||||||
|
discussion"* and that the rejected alternative lives "in issue #88's
|
||||||
|
comment thread." The PRD's rationale is therefore **incomplete without
|
||||||
|
the issue**. If Gitea is gone, the strongest argument for the chosen
|
||||||
|
design is gone with it.
|
||||||
|
- PR #89's description links `…/didericis/claude-bottle/issues/88` —
|
||||||
|
the **pre-rename** repo path (the project was Codex-bottle/claude-
|
||||||
|
bottle before the bot-bottle rebrand). That link is already
|
||||||
|
half-dead: a concrete demonstration that Gitea URLs rot under the
|
||||||
|
most routine event imaginable, a rename.
|
||||||
|
- Issue/PR numbers (`#88`, `#90`, `#94`, `#95`) are **Gitea-assigned
|
||||||
|
from a shared sequence**. They cannot be reconstructed from a clone,
|
||||||
|
and they collide/renumber on import into a different tracker.
|
||||||
|
|
||||||
|
So the question isn't academic. The current practice is already
|
||||||
|
producing references that don't survive a rename, let alone a migration.
|
||||||
|
|
||||||
|
## What each medium is actually good at
|
||||||
|
|
||||||
|
| Concern | Gitea issue | In-repo file (PRD / note / log) |
|
||||||
|
|---|---|---|
|
||||||
|
| Capture friction | Near-zero — file a one-line idea | High — a PRD is a heavy artifact; a note less so |
|
||||||
|
| Triage (labels, milestones, open/closed, assignee) | Native, good | Absent / hand-rolled |
|
||||||
|
| Async discussion + notifications | Native (threads, @mentions, watch) | None — needs a PR review or out-of-band chat |
|
||||||
|
| Auto-linking (`Closes #N`, PR↔issue, commit↔issue) | Native | Manual cross-reference |
|
||||||
|
| Version control of the content | None — lives in Gitea's DB | Full — diff, blame, branch, revert |
|
||||||
|
| Travels with `git clone` | No | Yes |
|
||||||
|
| Survives a move off Gitea | Degrades (export/import; threads, authors, timestamps, refs lossy) | Unaffected |
|
||||||
|
| Survives a Gitea outage | Inaccessible | Local clone has it |
|
||||||
|
| Greppable offline / by tooling | Only via API | `grep docs/` |
|
||||||
|
| Reproducible identifiers | Gitea-assigned numbers | Filenames you control (`0027-…`) |
|
||||||
|
|
||||||
|
The split is clean: **issues win on the live, social, coordination axes;
|
||||||
|
the repo wins on every durability and portability axis.** Nothing about
|
||||||
|
that table says "pick one." It says "use each for what it's good at, and
|
||||||
|
don't let the durable thing depend on the ephemeral one."
|
||||||
|
|
||||||
|
## Lock-in failure modes (the cons, concretely)
|
||||||
|
|
||||||
|
1. **Stranded rationale.** The single most valuable output of a feature
|
||||||
|
discussion — *why we rejected the obvious alternative* — usually
|
||||||
|
emerges in a thread and dies there unless someone copies it into the
|
||||||
|
spec. PRD 0025 is already in this state.
|
||||||
|
2. **Reference rot.** `Closes #88` / "see issue #90" are meaningful only
|
||||||
|
against one Gitea instance at one point in time. A rename already
|
||||||
|
broke one such link; a migration would break all of them and
|
||||||
|
silently renumber the survivors.
|
||||||
|
3. **Two sources of truth.** A PRD carries `Status: Draft`; the issue
|
||||||
|
carries open/closed. They drift. Which is authoritative?
|
||||||
|
4. **Availability coupling.** Self-hosted Gitea down (or the Tailscale
|
||||||
|
path to it down) means the backlog and its history are unreachable,
|
||||||
|
even though the code and PRDs are right there in the clone.
|
||||||
|
5. **Export is lossy.** Gitea→GitHub (or the reverse) moves issue *text*
|
||||||
|
tolerably but mangles cross-references, comment authorship for
|
||||||
|
non-mapped users, timestamps, and reactions. The graph of "#88 → PR
|
||||||
|
#89 → commit abc" does not survive intact.
|
||||||
|
|
||||||
|
None of these are arguments against *having* issues. They're arguments
|
||||||
|
against issues being the **only** place a decision is recorded.
|
||||||
|
|
||||||
|
## Pros of keeping issues anyway
|
||||||
|
|
||||||
|
Worth stating plainly, because "just use the repo for everything"
|
||||||
|
overcorrects:
|
||||||
|
|
||||||
|
- A PR per half-formed idea is absurd; issues are the right weight for
|
||||||
|
"someone should look at X someday."
|
||||||
|
- Triage state (priority, milestone, assignee, open/closed) is genuine
|
||||||
|
project-management value the repo does not natively provide.
|
||||||
|
- Notifications and threaded discussion are how a decision *gets made*
|
||||||
|
before it's ready to be written down. Killing issues doesn't move that
|
||||||
|
conversation into the repo — it moves it into chat/DMs, which is
|
||||||
|
*worse* for durability, not better.
|
||||||
|
- `Closes #N` automation and PR↔issue linkage are real ergonomics.
|
||||||
|
|
||||||
|
The goal is not to abandon the tracker. It's to make sure that when the
|
||||||
|
tracker eventually goes away, you lose the *backlog*, not the *history*.
|
||||||
|
|
||||||
|
## What belongs where
|
||||||
|
|
||||||
|
- **Gitea issue** — intake, triage, status, and the live discussion.
|
||||||
|
Treat it as a **cache**: useful now, expendable later.
|
||||||
|
- **PRD (`docs/prds/`)** — the durable spec for anything that warrants
|
||||||
|
one. Rule: a PRD must be **self-contained**. Synthesize the issue
|
||||||
|
discussion into the Problem / Design / Open-questions sections;
|
||||||
|
reference the issue as a convenience pointer, never as the only home
|
||||||
|
of a load-bearing argument. (Retrofit PRD 0025: inline the #88
|
||||||
|
"option 3 vs `bottle_config:`" reasoning so the PRD stands alone.)
|
||||||
|
- **Research note (`docs/research/`)** — the durable "why," exactly like
|
||||||
|
this file. Comparative analysis, landscape surveys, tradeoffs.
|
||||||
|
- **Commit message** — the durable "what changed and why, at this point
|
||||||
|
in the diff."
|
||||||
|
- **Decision log (proposed, see below)** — durable record of decisions
|
||||||
|
that aren't features and don't merit a PRD.
|
||||||
|
|
||||||
|
## Closing the gap: a portable decision record
|
||||||
|
|
||||||
|
Two classes of decision currently have no in-repo home:
|
||||||
|
|
||||||
|
- **Sub-PRD feature requests** — too small for a PRD, but you still want
|
||||||
|
a tracked "we will / won't do this, because." Today these live only as
|
||||||
|
issues.
|
||||||
|
- **Non-feature decisions** — "merge with rebase, not merge-commit,"
|
||||||
|
"agent identity is claimed-not-vouched," "bottles are home-only."
|
||||||
|
Some land inside a PRD that happens to touch them; many are folded
|
||||||
|
into chat and lost.
|
||||||
|
|
||||||
|
Options, cheapest first:
|
||||||
|
|
||||||
|
1. **An ADR-lite log under `docs/decisions/`.** One short Markdown file
|
||||||
|
per decision: context, decision, consequences, date, links. This is
|
||||||
|
the industry-standard Architecture Decision Record pattern, and it's
|
||||||
|
a near-exact fit for "track decision history, portably." Numbered
|
||||||
|
like PRDs (`0001-merge-with-rebase.md`). ~10 lines each; the
|
||||||
|
discipline is writing them, not the format.
|
||||||
|
2. **Reuse the journal.** The repo ships an `init-entry` skill that
|
||||||
|
writes timestamped prose to `docs/JOURNAL.md` (not yet created here).
|
||||||
|
A stream-of-thought journal is a fine home for decision *narrative*
|
||||||
|
and is already part of the toolchain — lower ceremony than ADRs, less
|
||||||
|
structured for later retrieval. The `tag-entries` skill could tag
|
||||||
|
decision entries for grep-ability.
|
||||||
|
3. **Periodic issue export.** Belt-and-suspenders: a scheduled job hits
|
||||||
|
the Gitea API and dumps open/closed issues + comments to JSON under
|
||||||
|
`docs/issues-archive/`, committed. Preserves the raw thread against
|
||||||
|
losing Gitea without changing daily workflow. Mechanical, not a
|
||||||
|
substitute for reifying rationale (a JSON dump of a thread is
|
||||||
|
evidence, not a decision).
|
||||||
|
|
||||||
|
These compose: ADRs/journal for the *decision*, optional export for the
|
||||||
|
*raw evidence*, issues for *live coordination*.
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
1. **Keep Gitea issues for intake, triage, and discussion.** Don't fight
|
||||||
|
Gitea on the things it's good at.
|
||||||
|
2. **Make the repo the system of record.** Adopt the rule: no decision
|
||||||
|
is "done" until its rationale exists in a versioned file (PRD,
|
||||||
|
research note, or decision log). The issue is a pointer, never the
|
||||||
|
sole source.
|
||||||
|
3. **Add `docs/decisions/` (ADR-lite).** Smallest change that closes the
|
||||||
|
real gap — sub-PRD requests and non-feature decisions. Start by
|
||||||
|
back-filling the few decisions already made only in threads or chat
|
||||||
|
(rebase-merge policy; the agent-identity trust call from PRD 0027).
|
||||||
|
4. **Retrofit PRD 0025** to inline its #88 rationale, removing the one
|
||||||
|
existing hard dependency on a Gitea thread.
|
||||||
|
5. **Treat issue numbers as disposable.** When a PRD/commit cites an
|
||||||
|
issue, ensure the cited content is mirrored in-repo so the citation
|
||||||
|
degrades to a dead-but-harmless link, not lost information. (The
|
||||||
|
already-broken `claude-bottle/issues/88` link is the warning.)
|
||||||
|
6. **Optional:** automate a Gitea issue export into the repo if you want
|
||||||
|
the raw threads preserved without manual transcription.
|
||||||
|
|
||||||
|
Net: issues stay, because the alternative to issues is chat, which is
|
||||||
|
worse. But the project's durable memory must live where the project
|
||||||
|
already lives — in the clone — so that moving off Gitea, or losing it,
|
||||||
|
costs you a backlog you can rebuild, never a history you can't.
|
||||||
@@ -148,7 +148,7 @@ telemetry to `statsig.anthropic.com` — are documented in
|
|||||||
[`agent-credential-proxy-landscape.md`](agent-credential-proxy-landscape.md)
|
[`agent-credential-proxy-landscape.md`](agent-credential-proxy-landscape.md)
|
||||||
§Anthropic / Claude Code.
|
§Anthropic / Claude Code.
|
||||||
|
|
||||||
**Forge-API gate (Gitea / GitHub / GitLab).** Holds the PAT;
|
**Git-host-API gate (Gitea / GitHub / GitLab).** Holds the PAT;
|
||||||
exposes a narrow REST surface. Token auth on all three is
|
exposes a narrow REST surface. Token auth on all three is
|
||||||
stateless `Authorization`-header injection — no CSRF, no request
|
stateless `Authorization`-header injection — no CSRF, no request
|
||||||
signing, no per-request nonce — so one proxy generalizes by
|
signing, no per-request nonce — so one proxy generalizes by
|
||||||
@@ -221,7 +221,7 @@ Add a `secret: true` flag (or a `secrets:` sibling of `env:`) that:
|
|||||||
AWS_SECRET_ACCESS_KEY").
|
AWS_SECRET_ACCESS_KEY").
|
||||||
- Refuses to launch if `egress.allowlist` contains any host that
|
- Refuses to launch if `egress.allowlist` contains any host that
|
||||||
is not source-controlled by the user (heuristic: not on a
|
is not source-controlled by the user (heuristic: not on a
|
||||||
built-in `KNOWN_FORGE_HOSTS` list).
|
built-in `KNOWN_GIT_HOSTS` list).
|
||||||
- Forces an explicit acknowledgement that a credential is being
|
- Forces an explicit acknowledgement that a credential is being
|
||||||
placed into the bottle rather than behind a gate.
|
placed into the bottle rather than behind a gate.
|
||||||
|
|
||||||
@@ -280,7 +280,7 @@ In priority order:
|
|||||||
([`agent-credential-proxy-landscape.md`](agent-credential-proxy-landscape.md)
|
([`agent-credential-proxy-landscape.md`](agent-credential-proxy-landscape.md)
|
||||||
§Recommended). Removes the highest-value secret and closes the
|
§Recommended). Removes the highest-value secret and closes the
|
||||||
passthrough hole as a side effect.
|
passthrough hole as a side effect.
|
||||||
2. **Forge-API gate** (same doc, same section — one proxy
|
2. **Git-host-API gate** (same doc, same section — one proxy
|
||||||
generalizes across Gitea / GitHub / GitLab by config).
|
generalizes across Gitea / GitHub / GitLab by config).
|
||||||
3. **Egress data budget** in pipelock — small lift, large damage
|
3. **Egress data budget** in pipelock — small lift, large damage
|
||||||
bound.
|
bound.
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ model: opus
|
|||||||
bottle: dev
|
bottle: dev
|
||||||
skills:
|
skills:
|
||||||
- init-prd
|
- init-prd
|
||||||
git:
|
git-gate:
|
||||||
user:
|
user:
|
||||||
name: implementer-bot
|
name: implementer-bot
|
||||||
email: eric+implementer@dideric.is
|
email: eric+implementer@dideric.is
|
||||||
|
|||||||
+11
-13
@@ -38,23 +38,21 @@ def fixture_with_egress_dict() -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
def fixture_with_git_dict() -> dict[str, Any]:
|
def fixture_with_git_dict() -> dict[str, Any]:
|
||||||
"""Bottle declares a git-gate upstream. JSON shape."""
|
"""Bottle declares git-gate upstreams. JSON shape."""
|
||||||
return {
|
return {
|
||||||
"bottles": {
|
"bottles": {
|
||||||
"dev": {
|
"dev": {
|
||||||
"git": {
|
"git-gate": {
|
||||||
"remotes": {
|
"repos": {
|
||||||
"gitea.dideric.is": {
|
"bot-bottle": {
|
||||||
"Name": "bot-bottle",
|
"url": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
|
||||||
"Upstream": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
|
"identity": "/dev/null",
|
||||||
"IdentityFile": "/dev/null",
|
"host_key": "ssh-ed25519 AAAA...",
|
||||||
"KnownHostKey": "ssh-ed25519 AAAA...",
|
|
||||||
},
|
},
|
||||||
"github.com": {
|
"foo": {
|
||||||
"Name": "foo",
|
"url": "ssh://git@github.com/didericis/foo.git",
|
||||||
"Upstream": "ssh://git@github.com/didericis/foo.git",
|
"identity": "/dev/null",
|
||||||
"IdentityFile": "/dev/null",
|
"host_key": "ssh-ed25519 BBBB...",
|
||||||
"KnownHostKey": "ssh-ed25519 BBBB...",
|
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,209 @@
|
|||||||
|
"""Unit: provider runtime defaults."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from bot_bottle.agent_provider import (
|
||||||
|
CODEX_HOST_CREDENTIAL_HOSTS,
|
||||||
|
agent_provision_plan,
|
||||||
|
runtime_for,
|
||||||
|
)
|
||||||
|
from bot_bottle.egress import CODEX_HOST_CREDENTIAL_TOKEN_REF
|
||||||
|
|
||||||
|
|
||||||
|
def _jwt(exp: int) -> str:
|
||||||
|
def enc(obj: dict) -> str:
|
||||||
|
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||||
|
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||||
|
return f"{enc({'alg': 'none'})}.{enc({'exp': exp})}.sig"
|
||||||
|
|
||||||
|
|
||||||
|
class TestAgentProviderRuntime(unittest.TestCase):
|
||||||
|
def test_codex_plan_declares_home_state(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="/tmp/Dockerfile.codex",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
)
|
||||||
|
config = Path(tmp, "codex-config.toml").read_text()
|
||||||
|
self.assertEqual("codex", plan.template)
|
||||||
|
self.assertEqual("codex", plan.command)
|
||||||
|
self.assertEqual("read_prompt_file", plan.prompt_mode)
|
||||||
|
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
|
||||||
|
self.assertEqual(
|
||||||
|
"/etc/ssl/certs/ca-certificates.crt",
|
||||||
|
plan.env_vars["CODEX_CA_CERTIFICATE"],
|
||||||
|
)
|
||||||
|
self.assertEqual({}, plan.guest_env)
|
||||||
|
self.assertEqual(("/home/node/.codex",), tuple(d.guest_path for d in plan.dirs))
|
||||||
|
self.assertEqual(
|
||||||
|
("/home/node/.codex/config.toml",),
|
||||||
|
tuple(f.guest_path for f in plan.files),
|
||||||
|
)
|
||||||
|
self.assertIn('[projects."/home/node"]', config)
|
||||||
|
|
||||||
|
def test_codex_trusts_requested_project_path(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
trusted_project_path="/home/node/workspace",
|
||||||
|
)
|
||||||
|
config = Path(tmp, "codex-config.toml").read_text()
|
||||||
|
self.assertIn('[projects."/home/node/workspace"]', config)
|
||||||
|
|
||||||
|
def test_codex_forward_host_credentials_adds_auth_and_verify(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
home = Path(tmp) / "host-codex"
|
||||||
|
home.mkdir()
|
||||||
|
(home / "auth.json").write_text(json.dumps({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": _jwt(2000000000)},
|
||||||
|
}))
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
guest_env={"CODEX_HOME": "/run/codex-home"},
|
||||||
|
forward_host_credentials=True,
|
||||||
|
host_env={"CODEX_HOME": str(home)},
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"/run/codex-home/auth.json",
|
||||||
|
{f.guest_path for f in plan.files},
|
||||||
|
)
|
||||||
|
self.assertEqual("/run/codex-home", plan.env_vars["CODEX_HOME"])
|
||||||
|
self.assertEqual(1, len(plan.pre_copy))
|
||||||
|
self.assertEqual(1, len(plan.verify))
|
||||||
|
self.assertIn("CODEX_HOME=/run/codex-home", plan.verify[0].argv)
|
||||||
|
|
||||||
|
def test_claude_with_auth_token_injects_provider_route_and_placeholder(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="claude",
|
||||||
|
dockerfile="/tmp/Dockerfile.claude",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
auth_token="BOT_BOTTLE_CLAUDE_OAUTH_TOKEN",
|
||||||
|
)
|
||||||
|
claude_config = json.loads(Path(tmp, "claude.json").read_text())
|
||||||
|
self.assertEqual(1, len(plan.egress_routes))
|
||||||
|
route = plan.egress_routes[0]
|
||||||
|
self.assertEqual("api.anthropic.com", route.host)
|
||||||
|
self.assertEqual("Bearer", route.auth_scheme)
|
||||||
|
self.assertEqual("BOT_BOTTLE_CLAUDE_OAUTH_TOKEN", route.token_ref)
|
||||||
|
self.assertTrue(route.tls_passthrough)
|
||||||
|
self.assertEqual("egress-placeholder", plan.env_vars["CLAUDE_CODE_OAUTH_TOKEN"])
|
||||||
|
self.assertEqual("1", plan.env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"])
|
||||||
|
self.assertEqual("1", plan.env_vars["DISABLE_ERROR_REPORTING"])
|
||||||
|
self.assertEqual(frozenset({"CLAUDE_CODE_OAUTH_TOKEN"}), plan.hidden_env_names)
|
||||||
|
self.assertIn("/home/node", claude_config["projects"])
|
||||||
|
self.assertIn("/home/node/.claude.json", {f.guest_path for f in plan.files})
|
||||||
|
|
||||||
|
def test_claude_trusts_requested_project_path(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
agent_provision_plan(
|
||||||
|
template="claude",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
trusted_project_path="/home/node/workspace",
|
||||||
|
)
|
||||||
|
config = json.loads(Path(tmp, "claude.json").read_text())
|
||||||
|
self.assertIn("/home/node", config["projects"])
|
||||||
|
self.assertIn("/home/node/workspace", config["projects"])
|
||||||
|
|
||||||
|
def test_codex_forward_host_credentials_populates_egress_routes(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
home = Path(tmp) / "host-codex"
|
||||||
|
home.mkdir()
|
||||||
|
(home / "auth.json").write_text(json.dumps({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": _jwt(2000000000)},
|
||||||
|
}))
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
forward_host_credentials=True,
|
||||||
|
host_env={"CODEX_HOME": str(home)},
|
||||||
|
)
|
||||||
|
hosts = [r.host for r in plan.egress_routes]
|
||||||
|
self.assertEqual(sorted(CODEX_HOST_CREDENTIAL_HOSTS), sorted(hosts))
|
||||||
|
for r in plan.egress_routes:
|
||||||
|
self.assertEqual("Bearer", r.auth_scheme)
|
||||||
|
self.assertEqual(CODEX_HOST_CREDENTIAL_TOKEN_REF, r.token_ref)
|
||||||
|
self.assertTrue(r.tls_passthrough)
|
||||||
|
|
||||||
|
def test_codex_without_forward_host_credentials_has_passthrough_egress_routes(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
forward_host_credentials=False,
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
{r.host for r in plan.egress_routes},
|
||||||
|
set(CODEX_HOST_CREDENTIAL_HOSTS),
|
||||||
|
)
|
||||||
|
for r in plan.egress_routes:
|
||||||
|
self.assertEqual("", r.auth_scheme)
|
||||||
|
self.assertEqual("", r.token_ref)
|
||||||
|
self.assertTrue(r.tls_passthrough)
|
||||||
|
|
||||||
|
def test_claude_without_auth_token_has_passthrough_egress_route(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="claude",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
)
|
||||||
|
self.assertEqual(1, len(plan.egress_routes))
|
||||||
|
route = plan.egress_routes[0]
|
||||||
|
self.assertEqual("api.anthropic.com", route.host)
|
||||||
|
self.assertEqual("", route.auth_scheme)
|
||||||
|
self.assertEqual("", route.token_ref)
|
||||||
|
self.assertTrue(route.tls_passthrough)
|
||||||
|
self.assertNotIn("CLAUDE_CODE_OAUTH_TOKEN", plan.env_vars)
|
||||||
|
self.assertEqual(frozenset(), plan.hidden_env_names)
|
||||||
|
|
||||||
|
def test_codex_forward_host_credentials_populates_provisioned_env(self):
|
||||||
|
access = _jwt(2000000000)
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
home = Path(tmp) / "host-codex"
|
||||||
|
home.mkdir()
|
||||||
|
(home / "auth.json").write_text(json.dumps({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": access},
|
||||||
|
}))
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
forward_host_credentials=True,
|
||||||
|
host_env={"CODEX_HOME": str(home)},
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
{CODEX_HOST_CREDENTIAL_TOKEN_REF: access},
|
||||||
|
plan.provisioned_env,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_codex_without_forward_host_credentials_has_empty_provisioned_env(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
|
||||||
|
plan = agent_provision_plan(
|
||||||
|
template="codex",
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=Path(tmp),
|
||||||
|
forward_host_credentials=False,
|
||||||
|
)
|
||||||
|
self.assertEqual({}, plan.provisioned_env)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,240 @@
|
|||||||
|
"""Cross-backend parity tests (PRD 0042).
|
||||||
|
|
||||||
|
Verifies that Docker and smolmachines bottles expose the same
|
||||||
|
observable contracts for env injection, agent argv, and exec. Tests
|
||||||
|
use mock subprocess layers so no live VM or Docker daemon is needed.
|
||||||
|
|
||||||
|
The scenarios here document what must hold across both backends. As
|
||||||
|
PRDs 0038–0040 land these tests provide regression coverage for the
|
||||||
|
contracts they establish.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import unittest
|
||||||
|
from typing import Callable
|
||||||
|
from unittest.mock import MagicMock, call, patch
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _docker_bottle(guest_env: dict[str, str]) -> "object":
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
return DockerBottle(
|
||||||
|
container="bot-bottle-test",
|
||||||
|
teardown=lambda: None,
|
||||||
|
prompt_path_in_container=None,
|
||||||
|
agent_command="claude",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
return SmolmachinesBottle(
|
||||||
|
"bot-bottle-test",
|
||||||
|
guest_env=guest_env,
|
||||||
|
agent_command="claude",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# One entry per backend: (label, factory).
|
||||||
|
_BACKENDS: list[tuple[str, Callable[[dict[str, str]], object]]] = [
|
||||||
|
("docker", _docker_bottle),
|
||||||
|
("smolmachines", _smolmachines_bottle),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# agent_argv contracts
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestAgentArgvParity(unittest.TestCase):
|
||||||
|
"""Both backends surface a non-empty agent_argv that includes the
|
||||||
|
agent command and can be used as a subprocess command list."""
|
||||||
|
|
||||||
|
def test_agent_argv_is_list_of_strings(self):
|
||||||
|
for label, factory in _BACKENDS:
|
||||||
|
with self.subTest(backend=label):
|
||||||
|
bottle = factory({"MY_VAR": "val"})
|
||||||
|
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
|
||||||
|
self.assertIsInstance(argv, list, f"{label}: argv is not a list")
|
||||||
|
for item in argv:
|
||||||
|
self.assertIsInstance(
|
||||||
|
item, str,
|
||||||
|
f"{label}: argv item {item!r} is not a str",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_agent_command_present_in_argv(self):
|
||||||
|
for label, factory in _BACKENDS:
|
||||||
|
with self.subTest(backend=label):
|
||||||
|
bottle = factory({})
|
||||||
|
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
|
||||||
|
joined = " ".join(argv)
|
||||||
|
self.assertIn(
|
||||||
|
"claude", joined,
|
||||||
|
f"{label}: 'claude' not found in agent_argv",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_extra_flags_propagate(self):
|
||||||
|
extra = ["--no-update-check", "--output-format", "stream-json"]
|
||||||
|
for label, factory in _BACKENDS:
|
||||||
|
with self.subTest(backend=label):
|
||||||
|
bottle = factory({})
|
||||||
|
argv = bottle.agent_argv(extra, tty=False) # type: ignore[union-attr]
|
||||||
|
for flag in extra:
|
||||||
|
self.assertIn(
|
||||||
|
flag, argv,
|
||||||
|
f"{label}: flag {flag!r} not in agent_argv",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestSmolmachinesEnvInArgv(unittest.TestCase):
|
||||||
|
"""smolmachines bottle includes guest_env values in exec argv."""
|
||||||
|
|
||||||
|
def test_guest_env_in_exec_argv(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
bottle = SmolmachinesBottle(
|
||||||
|
"bot-bottle-test",
|
||||||
|
guest_env={"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
|
||||||
|
)
|
||||||
|
argv = bottle.agent_argv([], tty=False)
|
||||||
|
joined = " ".join(argv)
|
||||||
|
self.assertIn("TOKEN=abc123", joined)
|
||||||
|
self.assertIn("PROXY=http://proxy:8888", joined)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# exec() user-switching contract
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestExecUserSwitching(unittest.TestCase):
|
||||||
|
"""Both backends exec as 'node' by default and accept user='root'."""
|
||||||
|
|
||||||
|
def test_docker_exec_uses_node_user_by_default(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
bottle = DockerBottle(
|
||||||
|
container="bot-bottle-test",
|
||||||
|
teardown=lambda: None,
|
||||||
|
prompt_path_in_container=None,
|
||||||
|
)
|
||||||
|
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
|
||||||
|
run.return_value = subprocess.CompletedProcess(
|
||||||
|
[], 0, stdout="", stderr="",
|
||||||
|
)
|
||||||
|
bottle.exec("echo hi")
|
||||||
|
call_args = run.call_args[0][0]
|
||||||
|
self.assertIn("node", call_args,
|
||||||
|
"docker exec should use 'node' user by default")
|
||||||
|
|
||||||
|
def test_smolmachines_exec_uses_node_user_by_default(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||||
|
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
|
||||||
|
run.return_value = subprocess.CompletedProcess(
|
||||||
|
[], 0, stdout="", stderr="",
|
||||||
|
)
|
||||||
|
bottle.exec("echo hi")
|
||||||
|
call_args = run.call_args[0][0]
|
||||||
|
self.assertIn("node", call_args,
|
||||||
|
"smolvm exec should use 'node' user by default")
|
||||||
|
|
||||||
|
def test_docker_exec_respects_root_user(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
bottle = DockerBottle(
|
||||||
|
container="bot-bottle-test",
|
||||||
|
teardown=lambda: None,
|
||||||
|
prompt_path_in_container=None,
|
||||||
|
)
|
||||||
|
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
|
||||||
|
run.return_value = subprocess.CompletedProcess(
|
||||||
|
[], 0, stdout="", stderr="",
|
||||||
|
)
|
||||||
|
bottle.exec("id", user="root")
|
||||||
|
call_args = run.call_args[0][0]
|
||||||
|
self.assertIn("root", call_args)
|
||||||
|
|
||||||
|
def test_smolmachines_exec_respects_root_user(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||||
|
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
|
||||||
|
run.return_value = subprocess.CompletedProcess(
|
||||||
|
[], 0, stdout="", stderr="",
|
||||||
|
)
|
||||||
|
bottle.exec("id", user="root")
|
||||||
|
call_args = run.call_args[0][0]
|
||||||
|
self.assertIn("root", call_args)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# ExecResult shape parity
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestExecResultParity(unittest.TestCase):
|
||||||
|
"""Both backends return ExecResult with returncode, stdout, stderr."""
|
||||||
|
|
||||||
|
def _stub_run(self, argv, **kwargs):
|
||||||
|
return subprocess.CompletedProcess(
|
||||||
|
argv, 0, stdout="out\n", stderr="err\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_docker_exec_result_shape(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
from bot_bottle.backend import ExecResult
|
||||||
|
bottle = DockerBottle(
|
||||||
|
container="bot-bottle-test",
|
||||||
|
teardown=lambda: None,
|
||||||
|
prompt_path_in_container=None,
|
||||||
|
)
|
||||||
|
with patch("bot_bottle.backend.docker.bottle.subprocess.run",
|
||||||
|
side_effect=self._stub_run):
|
||||||
|
result = bottle.exec("echo hi")
|
||||||
|
self.assertIsInstance(result, ExecResult)
|
||||||
|
self.assertEqual(0, result.returncode)
|
||||||
|
self.assertIsInstance(result.stdout, str)
|
||||||
|
self.assertIsInstance(result.stderr, str)
|
||||||
|
|
||||||
|
def test_smolmachines_exec_result_shape(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
from bot_bottle.backend import ExecResult
|
||||||
|
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||||
|
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run",
|
||||||
|
side_effect=self._stub_run):
|
||||||
|
result = bottle.exec("echo hi")
|
||||||
|
self.assertIsInstance(result, ExecResult)
|
||||||
|
self.assertEqual(0, result.returncode)
|
||||||
|
self.assertIsInstance(result.stdout, str)
|
||||||
|
self.assertIsInstance(result.stderr, str)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# close() is a no-op / idempotent (ABC contract)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestCloseParity(unittest.TestCase):
|
||||||
|
def test_docker_close_is_idempotent(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
teardown_count = [0]
|
||||||
|
def count_teardown():
|
||||||
|
teardown_count[0] += 1
|
||||||
|
bottle = DockerBottle(
|
||||||
|
container="bot-bottle-test",
|
||||||
|
teardown=count_teardown,
|
||||||
|
prompt_path_in_container=None,
|
||||||
|
)
|
||||||
|
bottle.close()
|
||||||
|
bottle.close()
|
||||||
|
# DockerBottle.close calls teardown — once per call is fine;
|
||||||
|
# what matters is it doesn't raise.
|
||||||
|
|
||||||
|
def test_smolmachines_close_is_noop(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||||
|
bottle.close()
|
||||||
|
bottle.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -81,6 +81,46 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
|||||||
):
|
):
|
||||||
self.assertEqual([a, b], enumerate_active_agents())
|
self.assertEqual([a, b], enumerate_active_agents())
|
||||||
|
|
||||||
|
def test_sorts_by_started_at_then_slug_across_backends(self):
|
||||||
|
newer = ActiveAgent(
|
||||||
|
backend_name="docker", slug="docker-new", agent_name="impl",
|
||||||
|
started_at="2026-06-02T12:00:00Z", services=(),
|
||||||
|
)
|
||||||
|
tie_b = ActiveAgent(
|
||||||
|
backend_name="docker", slug="b-slug", agent_name="review",
|
||||||
|
started_at="2026-06-02T11:00:00Z", services=(),
|
||||||
|
)
|
||||||
|
missing_metadata = ActiveAgent(
|
||||||
|
backend_name="smolmachines", slug="missing-metadata",
|
||||||
|
agent_name="?", started_at="", services=(),
|
||||||
|
)
|
||||||
|
tie_a = ActiveAgent(
|
||||||
|
backend_name="smolmachines", slug="a-slug", agent_name="research",
|
||||||
|
started_at="2026-06-02T11:00:00Z", services=(),
|
||||||
|
)
|
||||||
|
|
||||||
|
class _FakeBackend:
|
||||||
|
def __init__(self, items):
|
||||||
|
self._items = items
|
||||||
|
|
||||||
|
def is_available(self):
|
||||||
|
return True
|
||||||
|
|
||||||
|
def enumerate_active(self):
|
||||||
|
return self._items
|
||||||
|
|
||||||
|
with patch.object(
|
||||||
|
backend_mod, "_BACKENDS",
|
||||||
|
{
|
||||||
|
"docker": _FakeBackend([newer, tie_b]),
|
||||||
|
"smolmachines": _FakeBackend([missing_metadata, tie_a]),
|
||||||
|
},
|
||||||
|
):
|
||||||
|
self.assertEqual(
|
||||||
|
[missing_metadata, tie_a, tie_b, newer],
|
||||||
|
enumerate_active_agents(),
|
||||||
|
)
|
||||||
|
|
||||||
def test_empty_when_no_backends_have_active(self):
|
def test_empty_when_no_backends_have_active(self):
|
||||||
class _FakeBackend:
|
class _FakeBackend:
|
||||||
def is_available(self):
|
def is_available(self):
|
||||||
|
|||||||
@@ -216,5 +216,112 @@ class TestBottleMetadata(_FakeHomeMixin, unittest.TestCase):
|
|||||||
self.assertEqual("t2", loaded.started_at)
|
self.assertEqual("t2", loaded.started_at)
|
||||||
|
|
||||||
|
|
||||||
|
class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
|
||||||
|
"""PRD 0040: backend field is persisted and read back."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self._setup_fake_home()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self._teardown_fake_home()
|
||||||
|
|
||||||
|
def test_backend_field_roundtrips_docker(self):
|
||||||
|
meta = BottleMetadata(
|
||||||
|
identity="dev-b1",
|
||||||
|
agent_name="dev",
|
||||||
|
cwd="",
|
||||||
|
copy_cwd=False,
|
||||||
|
started_at="2026-06-02T00:00:00+00:00",
|
||||||
|
compose_project="bot-bottle-dev-b1",
|
||||||
|
backend="docker",
|
||||||
|
)
|
||||||
|
write_metadata(meta)
|
||||||
|
loaded = read_metadata("dev-b1")
|
||||||
|
self.assertIsNotNone(loaded)
|
||||||
|
assert loaded is not None
|
||||||
|
self.assertEqual("docker", loaded.backend)
|
||||||
|
|
||||||
|
def test_backend_field_roundtrips_smolmachines(self):
|
||||||
|
meta = BottleMetadata(
|
||||||
|
identity="dev-b2",
|
||||||
|
agent_name="dev",
|
||||||
|
cwd="",
|
||||||
|
copy_cwd=False,
|
||||||
|
started_at="2026-06-02T00:00:00+00:00",
|
||||||
|
compose_project="",
|
||||||
|
backend="smolmachines",
|
||||||
|
)
|
||||||
|
write_metadata(meta)
|
||||||
|
loaded = read_metadata("dev-b2")
|
||||||
|
self.assertIsNotNone(loaded)
|
||||||
|
assert loaded is not None
|
||||||
|
self.assertEqual("smolmachines", loaded.backend)
|
||||||
|
|
||||||
|
def test_missing_backend_field_defaults_to_empty(self):
|
||||||
|
# Old state dirs written before PRD 0040 have no backend key.
|
||||||
|
import json
|
||||||
|
from bot_bottle.backend.docker import bottle_state as bs
|
||||||
|
path = bs.metadata_path("dev-b3")
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(json.dumps({
|
||||||
|
"identity": "dev-b3",
|
||||||
|
"agent_name": "dev",
|
||||||
|
"cwd": "",
|
||||||
|
"copy_cwd": False,
|
||||||
|
"started_at": "2026-06-02T00:00:00+00:00",
|
||||||
|
"compose_project": "bot-bottle-dev-b3",
|
||||||
|
}))
|
||||||
|
loaded = read_metadata("dev-b3")
|
||||||
|
self.assertIsNotNone(loaded)
|
||||||
|
assert loaded is not None
|
||||||
|
self.assertEqual("", loaded.backend)
|
||||||
|
|
||||||
|
|
||||||
|
class TestBottleForSlugBackend(_FakeHomeMixin, unittest.TestCase):
|
||||||
|
"""PRD 0040: _bottle_for_slug constructs the right bottle type."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self._setup_fake_home()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self._teardown_fake_home()
|
||||||
|
|
||||||
|
def test_docker_metadata_returns_docker_bottle(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
from bot_bottle.cli.dashboard import _bottle_for_slug
|
||||||
|
write_metadata(BottleMetadata(
|
||||||
|
identity="dev-d1",
|
||||||
|
agent_name="dev",
|
||||||
|
cwd="",
|
||||||
|
copy_cwd=False,
|
||||||
|
started_at="2026-06-02T00:00:00+00:00",
|
||||||
|
compose_project="bot-bottle-dev-d1",
|
||||||
|
backend="docker",
|
||||||
|
))
|
||||||
|
bottle, _ = _bottle_for_slug("dev-d1", {}, None)
|
||||||
|
self.assertIsInstance(bottle, DockerBottle)
|
||||||
|
|
||||||
|
def test_smolmachines_metadata_returns_smolmachines_bottle(self):
|
||||||
|
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||||
|
from bot_bottle.cli.dashboard import _bottle_for_slug
|
||||||
|
write_metadata(BottleMetadata(
|
||||||
|
identity="dev-s1",
|
||||||
|
agent_name="dev",
|
||||||
|
cwd="",
|
||||||
|
copy_cwd=False,
|
||||||
|
started_at="2026-06-02T00:00:00+00:00",
|
||||||
|
compose_project="",
|
||||||
|
backend="smolmachines",
|
||||||
|
))
|
||||||
|
bottle, _ = _bottle_for_slug("dev-s1", {}, None)
|
||||||
|
self.assertIsInstance(bottle, SmolmachinesBottle)
|
||||||
|
|
||||||
|
def test_no_metadata_defaults_to_docker_bottle(self):
|
||||||
|
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||||
|
from bot_bottle.cli.dashboard import _bottle_for_slug
|
||||||
|
bottle, _ = _bottle_for_slug("unknown-slug", {}, None)
|
||||||
|
self.assertIsInstance(bottle, DockerBottle)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -0,0 +1,297 @@
|
|||||||
|
"""Unit: host Codex auth extraction."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from bot_bottle.codex_auth import (
|
||||||
|
codex_auth_path,
|
||||||
|
codex_dummy_auth_json,
|
||||||
|
codex_host_access_token,
|
||||||
|
)
|
||||||
|
from bot_bottle.log import Die
|
||||||
|
|
||||||
|
|
||||||
|
def _jwt(exp: int) -> str:
|
||||||
|
return _jwt_with_payload({"exp": exp})
|
||||||
|
|
||||||
|
|
||||||
|
def _jwt_with_payload(payload: dict) -> str:
|
||||||
|
def enc(obj: dict) -> str:
|
||||||
|
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||||
|
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||||
|
return f"{enc({'alg': 'none'})}.{enc(payload)}.sig"
|
||||||
|
|
||||||
|
|
||||||
|
def _jwt_payload(token: str) -> dict:
|
||||||
|
payload = token.split(".")[1]
|
||||||
|
payload += "=" * (-len(payload) % 4)
|
||||||
|
return json.loads(base64.urlsafe_b64decode(payload.encode()).decode())
|
||||||
|
|
||||||
|
|
||||||
|
class TestCodexHostAccessToken(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.tmp = tempfile.TemporaryDirectory(prefix="cb-codex-auth.")
|
||||||
|
self.home = Path(self.tmp.name)
|
||||||
|
self.auth_path = self.home / "auth.json"
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.tmp.cleanup()
|
||||||
|
|
||||||
|
def _write(self, payload: dict) -> None:
|
||||||
|
self.auth_path.write_text(json.dumps(payload))
|
||||||
|
|
||||||
|
def test_auth_path_uses_codex_home(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.auth_path,
|
||||||
|
codex_auth_path({"CODEX_HOME": str(self.home)}),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_returns_fresh_chatgpt_access_token(self):
|
||||||
|
token = _jwt(2000000000)
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": token, "refresh_token": "hidden"},
|
||||||
|
})
|
||||||
|
out = codex_host_access_token(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
self.assertEqual(token, out)
|
||||||
|
|
||||||
|
def test_missing_auth_file_dies(self):
|
||||||
|
with self.assertRaises(Die):
|
||||||
|
codex_host_access_token({"CODEX_HOME": str(self.home)})
|
||||||
|
|
||||||
|
def test_non_chatgpt_auth_dies(self):
|
||||||
|
self._write({"auth_mode": "api_key", "tokens": {"access_token": _jwt(2)}})
|
||||||
|
with self.assertRaises(Die):
|
||||||
|
codex_host_access_token({"CODEX_HOME": str(self.home)})
|
||||||
|
|
||||||
|
def test_user_auth_mode_is_allowed(self):
|
||||||
|
token = _jwt(2000000000)
|
||||||
|
self._write({"auth_mode": "user", "tokens": {"access_token": token}})
|
||||||
|
out = codex_host_access_token(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
self.assertEqual(token, out)
|
||||||
|
|
||||||
|
def test_expired_token_dies(self):
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": _jwt(1000)},
|
||||||
|
})
|
||||||
|
with self.assertRaises(Die):
|
||||||
|
codex_host_access_token(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_non_jwt_token_dies(self):
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {"access_token": "not-a-jwt"},
|
||||||
|
})
|
||||||
|
with self.assertRaises(Die):
|
||||||
|
codex_host_access_token({"CODEX_HOME": str(self.home)})
|
||||||
|
|
||||||
|
def test_dummy_auth_preserves_mode_and_redacts_tokens(self):
|
||||||
|
access = _jwt(2000000000)
|
||||||
|
refresh = "host-refresh-token"
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"OPENAI_API_KEY": None,
|
||||||
|
"tokens": {
|
||||||
|
"access_token": access,
|
||||||
|
"id_token": _jwt(2000000000),
|
||||||
|
"refresh_token": refresh,
|
||||||
|
"account_id": "acct-host",
|
||||||
|
},
|
||||||
|
"last_refresh": "2026-05-29T00:00:00.000Z",
|
||||||
|
})
|
||||||
|
dummy = json.loads(codex_dummy_auth_json(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
))
|
||||||
|
self.assertEqual("chatgpt", dummy["auth_mode"])
|
||||||
|
self.assertIsNone(dummy["OPENAI_API_KEY"])
|
||||||
|
self.assertNotEqual(access, dummy["tokens"]["access_token"])
|
||||||
|
self.assertNotEqual(refresh, dummy["tokens"]["refresh_token"])
|
||||||
|
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["refresh_token"])
|
||||||
|
self.assertEqual("acct-host", dummy["tokens"]["account_id"])
|
||||||
|
self.assertIsNotNone(
|
||||||
|
codex_host_access_token(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_dummy_auth_tokens_inherit_host_token_exp(self):
|
||||||
|
# Codex refreshes when its local access token is at/past exp;
|
||||||
|
# the dummy must carry the host token's real exp so Codex does
|
||||||
|
# not drop to the sign-in screen after an artificial TTL while
|
||||||
|
# egress still holds a valid bearer.
|
||||||
|
host_exp = 2000000000
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {
|
||||||
|
"access_token": _jwt(host_exp),
|
||||||
|
"id_token": _jwt(host_exp),
|
||||||
|
"refresh_token": "hidden",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
dummy = json.loads(codex_dummy_auth_json(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
))
|
||||||
|
self.assertEqual(
|
||||||
|
host_exp, _jwt_payload(dummy["tokens"]["access_token"])["exp"],
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
host_exp, _jwt_payload(dummy["tokens"]["id_token"])["exp"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_dummy_auth_replaces_last_refresh_with_valid_timestamp(self):
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"last_refresh": "host-refresh-metadata",
|
||||||
|
"tokens": {
|
||||||
|
"access_token": _jwt(2000000000),
|
||||||
|
"refresh_token": "hidden",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
dummy = json.loads(codex_dummy_auth_json(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, 2, 3, 4, 5000, tzinfo=timezone.utc),
|
||||||
|
))
|
||||||
|
self.assertEqual("2026-01-01T02:03:04.005Z", dummy["last_refresh"])
|
||||||
|
self.assertNotEqual("host-refresh-metadata", dummy["last_refresh"])
|
||||||
|
|
||||||
|
def test_dummy_auth_keeps_required_account_claim_shape(self):
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"tokens": {
|
||||||
|
"access_token": _jwt_with_payload({
|
||||||
|
"exp": 2000000000,
|
||||||
|
"https://api.openai.com/auth": {
|
||||||
|
"chatgpt_plan_type": "plus",
|
||||||
|
"chatgpt_account_id": "acct-real",
|
||||||
|
"chatgpt_user_id": "user-real",
|
||||||
|
"user_id": "auth-user-real",
|
||||||
|
"localhost": True,
|
||||||
|
},
|
||||||
|
"https://api.openai.com/profile": {
|
||||||
|
"email": "real@example.invalid",
|
||||||
|
"email_verified": True,
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
"id_token": _jwt_with_payload({
|
||||||
|
"exp": 2000000000,
|
||||||
|
"email": "real@example.invalid",
|
||||||
|
"email_verified": True,
|
||||||
|
"https://api.openai.com/auth": {
|
||||||
|
"chatgpt_plan_type": "plus",
|
||||||
|
"chatgpt_account_id": "acct-real",
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
"refresh_token": "hidden",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
dummy = json.loads(codex_dummy_auth_json(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
))
|
||||||
|
access_payload = _jwt_payload(dummy["tokens"]["access_token"])
|
||||||
|
auth = access_payload["https://api.openai.com/auth"]
|
||||||
|
profile = access_payload["https://api.openai.com/profile"]
|
||||||
|
self.assertEqual("plus", auth["chatgpt_plan_type"])
|
||||||
|
self.assertEqual("acct-real", auth["chatgpt_account_id"])
|
||||||
|
self.assertEqual("bot-bottle-placeholder", auth["chatgpt_user_id"])
|
||||||
|
self.assertEqual("bot-bottle@example.invalid", profile["email"])
|
||||||
|
self.assertTrue(profile["email_verified"])
|
||||||
|
|
||||||
|
def test_dummy_auth_redacts_unknown_future_auth_fields(self):
|
||||||
|
secrets = [
|
||||||
|
"top-session-secret",
|
||||||
|
"top-nested-secret",
|
||||||
|
"refresh-secret",
|
||||||
|
"session-token-secret",
|
||||||
|
"jwt-custom-secret",
|
||||||
|
"jwt-nested-secret",
|
||||||
|
"jwt-list-secret",
|
||||||
|
"id-token-secret",
|
||||||
|
"auth-claim-secret",
|
||||||
|
"auth-claim-nested-secret",
|
||||||
|
"top-list-secret",
|
||||||
|
"token-nested-secret",
|
||||||
|
"token-list-secret",
|
||||||
|
"last-refresh-secret",
|
||||||
|
]
|
||||||
|
self._write({
|
||||||
|
"auth_mode": "chatgpt",
|
||||||
|
"session_context": "top-session-secret",
|
||||||
|
"last_refresh": "last-refresh-secret",
|
||||||
|
"future_nested": {"value": "top-nested-secret"},
|
||||||
|
"future_list": ["top-list-secret"],
|
||||||
|
"tokens": {
|
||||||
|
"access_token": _jwt_with_payload({
|
||||||
|
"exp": 2000000000,
|
||||||
|
"custom_session": "jwt-custom-secret",
|
||||||
|
"future_nested": {"value": "jwt-nested-secret"},
|
||||||
|
"future_list": ["jwt-list-secret"],
|
||||||
|
"https://api.openai.com/auth": {
|
||||||
|
"chatgpt_plan_type": "plus",
|
||||||
|
"chatgpt_account_id": "acct-real",
|
||||||
|
"session_context": "auth-claim-secret",
|
||||||
|
"nested": {"value": "auth-claim-nested-secret"},
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
"id_token": _jwt_with_payload({
|
||||||
|
"exp": 2000000000,
|
||||||
|
"opaque": "id-token-secret",
|
||||||
|
}),
|
||||||
|
"refresh_token": "refresh-secret",
|
||||||
|
"session_token": "session-token-secret",
|
||||||
|
"future_object": {"value": "token-nested-secret"},
|
||||||
|
"future_list": ["token-list-secret"],
|
||||||
|
"account_id": "acct-host",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
dummy_json = codex_dummy_auth_json(
|
||||||
|
{"CODEX_HOME": str(self.home)},
|
||||||
|
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
for secret in secrets:
|
||||||
|
self.assertNotIn(secret, dummy_json)
|
||||||
|
|
||||||
|
dummy = json.loads(dummy_json)
|
||||||
|
self.assertEqual("bot-bottle-placeholder", dummy["session_context"])
|
||||||
|
self.assertEqual("2026-01-01T00:00:00.000Z", dummy["last_refresh"])
|
||||||
|
self.assertEqual({}, dummy["future_nested"])
|
||||||
|
self.assertEqual([], dummy["future_list"])
|
||||||
|
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["refresh_token"])
|
||||||
|
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["session_token"])
|
||||||
|
self.assertEqual({}, dummy["tokens"]["future_object"])
|
||||||
|
self.assertEqual([], dummy["tokens"]["future_list"])
|
||||||
|
|
||||||
|
access_payload = _jwt_payload(dummy["tokens"]["access_token"])
|
||||||
|
self.assertEqual(
|
||||||
|
"bot-bottle-placeholder",
|
||||||
|
access_payload["custom_session"],
|
||||||
|
)
|
||||||
|
self.assertEqual({}, access_payload["future_nested"])
|
||||||
|
self.assertEqual([], access_payload["future_list"])
|
||||||
|
auth = access_payload["https://api.openai.com/auth"]
|
||||||
|
self.assertEqual("bot-bottle-placeholder", auth["session_context"])
|
||||||
|
self.assertEqual({}, auth["nested"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
+32
-12
@@ -14,6 +14,7 @@ import unittest
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
|
from bot_bottle.agent_provider import AgentProvisionPlan
|
||||||
from bot_bottle.backend import BottleSpec
|
from bot_bottle.backend import BottleSpec
|
||||||
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||||
from bot_bottle.backend.docker.compose import (
|
from bot_bottle.backend.docker.compose import (
|
||||||
@@ -32,6 +33,7 @@ from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
|
|||||||
from bot_bottle.manifest import Manifest
|
from bot_bottle.manifest import Manifest
|
||||||
from bot_bottle.pipelock import PipelockProxyPlan
|
from bot_bottle.pipelock import PipelockProxyPlan
|
||||||
from bot_bottle.supervise import SupervisePlan
|
from bot_bottle.supervise import SupervisePlan
|
||||||
|
from bot_bottle.workspace import workspace_plan
|
||||||
|
|
||||||
|
|
||||||
SLUG = "demo-abc12"
|
SLUG = "demo-abc12"
|
||||||
@@ -47,11 +49,10 @@ def _manifest(*, supervise: bool, with_git: bool, with_egress: bool) -> Manifest
|
|||||||
if supervise:
|
if supervise:
|
||||||
bottle["supervise"] = True
|
bottle["supervise"] = True
|
||||||
if with_git:
|
if with_git:
|
||||||
bottle["git"] = {"remotes": {
|
bottle["git-gate"] = {"repos": {
|
||||||
"example.com": {
|
"upstream": {
|
||||||
"Name": "upstream",
|
"url": "ssh://git@example.com:22/x/y.git",
|
||||||
"Upstream": "ssh://git@example.com:22/x/y.git",
|
"identity": "/etc/hostname", # any existing file
|
||||||
"IdentityFile": "/etc/hostname", # any existing file
|
|
||||||
},
|
},
|
||||||
}}
|
}}
|
||||||
if with_egress:
|
if with_egress:
|
||||||
@@ -149,7 +150,6 @@ def _plan(
|
|||||||
identity_file="/etc/hostname",
|
identity_file="/etc/hostname",
|
||||||
known_host_key="",
|
known_host_key="",
|
||||||
known_hosts_file=STATE / "git-gate" / "upstream-known_hosts",
|
known_hosts_file=STATE / "git-gate" / "upstream-known_hosts",
|
||||||
extra_hosts={"example.com": "10.0.0.1"},
|
|
||||||
),)
|
),)
|
||||||
routes: tuple[EgressRoute, ...] = ()
|
routes: tuple[EgressRoute, ...] = ()
|
||||||
if with_egress:
|
if with_egress:
|
||||||
@@ -162,8 +162,9 @@ def _plan(
|
|||||||
roles=(),
|
roles=(),
|
||||||
),)
|
),)
|
||||||
|
|
||||||
|
spec = _spec(supervise=supervise, with_git=with_git, with_egress=with_egress)
|
||||||
return DockerBottlePlan(
|
return DockerBottlePlan(
|
||||||
spec=_spec(supervise=supervise, with_git=with_git, with_egress=with_egress),
|
spec=spec,
|
||||||
stage_dir=STAGE,
|
stage_dir=STAGE,
|
||||||
slug=SLUG,
|
slug=SLUG,
|
||||||
container_name=f"bot-bottle-{SLUG}",
|
container_name=f"bot-bottle-{SLUG}",
|
||||||
@@ -180,6 +181,15 @@ def _plan(
|
|||||||
egress_plan=_egress_plan(routes),
|
egress_plan=_egress_plan(routes),
|
||||||
supervise_plan=_supervise_plan() if supervise else None,
|
supervise_plan=_supervise_plan() if supervise else None,
|
||||||
use_runsc=False,
|
use_runsc=False,
|
||||||
|
agent_provision=AgentProvisionPlan(
|
||||||
|
template="claude",
|
||||||
|
command="claude",
|
||||||
|
prompt_mode="append_file",
|
||||||
|
image="bot-bottle-claude:latest",
|
||||||
|
dockerfile="",
|
||||||
|
guest_env={},
|
||||||
|
),
|
||||||
|
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -250,6 +260,20 @@ class TestAgentAlwaysPresent(unittest.TestCase):
|
|||||||
s = bottle_plan_to_compose(_plan())["services"]["agent"]
|
s = bottle_plan_to_compose(_plan())["services"]["agent"]
|
||||||
self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", s["environment"])
|
self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", s["environment"])
|
||||||
|
|
||||||
|
def test_agent_provider_env_uses_literal_values(self):
|
||||||
|
plan = _plan()
|
||||||
|
provision = AgentProvisionPlan(
|
||||||
|
template="codex",
|
||||||
|
command="codex",
|
||||||
|
prompt_mode="read_prompt_file",
|
||||||
|
image="bot-bottle-codex:latest",
|
||||||
|
dockerfile="",
|
||||||
|
guest_env={"CODEX_HOME": "/home/node/.codex"},
|
||||||
|
)
|
||||||
|
plan = type(plan)(**{**vars(plan), "agent_provision": provision})
|
||||||
|
s = bottle_plan_to_compose(plan)["services"]["agent"]
|
||||||
|
self.assertIn("CODEX_HOME=/home/node/.codex", s["environment"])
|
||||||
|
|
||||||
def test_agent_runsc_runtime(self):
|
def test_agent_runsc_runtime(self):
|
||||||
plan = _plan()
|
plan = _plan()
|
||||||
plan = type(plan)(**{**vars(plan), "use_runsc": True})
|
plan = type(plan)(**{**vars(plan), "use_runsc": True})
|
||||||
@@ -414,12 +438,8 @@ class TestSidecarBundleShape(unittest.TestCase):
|
|||||||
self.assertTrue(any("supervise/queue" in t or t.startswith("/run/supervise")
|
self.assertTrue(any("supervise/queue" in t or t.startswith("/run/supervise")
|
||||||
for t in targets))
|
for t in targets))
|
||||||
|
|
||||||
def test_extra_hosts_emitted_for_git_upstreams(self):
|
def test_extra_hosts_omitted_for_git_upstreams(self):
|
||||||
sc = self._render(with_git=True)["services"]["sidecars"]
|
sc = self._render(with_git=True)["services"]["sidecars"]
|
||||||
self.assertIn("example.com:10.0.0.1", sc.get("extra_hosts", []))
|
|
||||||
|
|
||||||
def test_extra_hosts_omitted_when_no_git(self):
|
|
||||||
sc = self._render()["services"]["sidecars"]
|
|
||||||
self.assertNotIn("extra_hosts", sc)
|
self.assertNotIn("extra_hosts", sc)
|
||||||
|
|
||||||
def test_agent_depends_on_bundle_only(self):
|
def test_agent_depends_on_bundle_only(self):
|
||||||
|
|||||||
@@ -577,5 +577,54 @@ class TestEditInEditor(unittest.TestCase):
|
|||||||
os.environ["EDITOR"] = original_editor
|
os.environ["EDITOR"] = original_editor
|
||||||
|
|
||||||
|
|
||||||
|
class TestCapabilityBlockSmolmachinesGuard(_FakeHomeMixin, unittest.TestCase):
|
||||||
|
"""approve() must refuse capability-block for smolmachines bottles and
|
||||||
|
pass it through for Docker bottles (PRD 0039)."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self._setup_fake_home()
|
||||||
|
self._original_apply_capability = dashboard.apply_capability_change
|
||||||
|
dashboard.apply_capability_change = lambda slug, content: ("", content)
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
dashboard.apply_capability_change = self._original_apply_capability
|
||||||
|
self._teardown_fake_home()
|
||||||
|
|
||||||
|
def _enqueue_capability(self, slug: str = "dev") -> "dashboard.QueuedProposal":
|
||||||
|
p = _proposal(slug=slug, tool=TOOL_CAPABILITY_BLOCK)
|
||||||
|
qdir = supervise.queue_dir_for_slug(slug)
|
||||||
|
qdir.mkdir(parents=True, exist_ok=True)
|
||||||
|
supervise.write_proposal(qdir, p)
|
||||||
|
return dashboard.QueuedProposal(proposal=p, queue_dir=qdir)
|
||||||
|
|
||||||
|
def _write_metadata(self, slug: str, compose_project: str) -> None:
|
||||||
|
from bot_bottle.backend.docker.bottle_state import BottleMetadata, write_metadata
|
||||||
|
write_metadata(BottleMetadata(
|
||||||
|
identity=slug,
|
||||||
|
agent_name="myagent",
|
||||||
|
cwd="",
|
||||||
|
copy_cwd=False,
|
||||||
|
started_at="2026-06-02T00:00:00+00:00",
|
||||||
|
compose_project=compose_project,
|
||||||
|
))
|
||||||
|
|
||||||
|
def test_smolmachines_bottle_raises_capability_apply_error(self):
|
||||||
|
self._write_metadata("dev", compose_project="")
|
||||||
|
qp = self._enqueue_capability("dev")
|
||||||
|
with self.assertRaises(CapabilityApplyError) as ctx:
|
||||||
|
dashboard.approve(qp)
|
||||||
|
self.assertIn("smolmachines", str(ctx.exception))
|
||||||
|
|
||||||
|
def test_docker_bottle_calls_apply_capability_change(self):
|
||||||
|
self._write_metadata("dev", compose_project="bot-bottle-dev")
|
||||||
|
qp = self._enqueue_capability("dev")
|
||||||
|
dashboard.approve(qp) # must not raise
|
||||||
|
|
||||||
|
def test_no_metadata_falls_through_to_docker_path(self):
|
||||||
|
# No metadata at all → assume Docker (backward-compatible).
|
||||||
|
qp = self._enqueue_capability("dev")
|
||||||
|
dashboard.approve(qp) # must not raise
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -0,0 +1,140 @@
|
|||||||
|
"""Unit: dashboard launch/crash failure logging (issue #100).
|
||||||
|
|
||||||
|
The dashboard runs under curses, so anything written to stderr while the
|
||||||
|
TUI owns the terminal is wiped when the terminal is restored. These
|
||||||
|
tests lock the recovery paths: a config error (`Die`) is re-surfaced
|
||||||
|
after the wrapper returns, and an unexpected crash is persisted to a
|
||||||
|
log file the operator can read.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import io
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from bot_bottle import supervise
|
||||||
|
from bot_bottle.cli import dashboard
|
||||||
|
from bot_bottle.log import Die, die
|
||||||
|
|
||||||
|
|
||||||
|
class TestDieCarriesMessage(unittest.TestCase):
|
||||||
|
def test_die_attaches_message_and_code(self):
|
||||||
|
buf = io.StringIO()
|
||||||
|
with contextlib.redirect_stderr(buf):
|
||||||
|
with self.assertRaises(Die) as cm:
|
||||||
|
die("bad manifest: unknown key 'foo'")
|
||||||
|
self.assertEqual("bad manifest: unknown key 'foo'", cm.exception.message)
|
||||||
|
self.assertEqual(1, cm.exception.code)
|
||||||
|
self.assertIn(
|
||||||
|
"bot-bottle: error: bad manifest: unknown key 'foo'", buf.getvalue()
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_die_default_message_is_empty(self):
|
||||||
|
self.assertEqual("", Die(1).message)
|
||||||
|
self.assertEqual(1, Die(1).code)
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeHomeMixin:
|
||||||
|
"""Point supervise.bot_bottle_root (what _write_crash_log resolves
|
||||||
|
through) at a temp dir so the crash log doesn't touch the real
|
||||||
|
~/.bot-bottle."""
|
||||||
|
|
||||||
|
def _setup_fake_home(self):
|
||||||
|
self._tmp = tempfile.TemporaryDirectory(prefix="dash-crash-test.")
|
||||||
|
self._orig_root = supervise.bot_bottle_root
|
||||||
|
self._root = Path(self._tmp.name) / ".bot-bottle"
|
||||||
|
supervise.bot_bottle_root = lambda: self._root # type: ignore[assignment]
|
||||||
|
|
||||||
|
def _teardown_fake_home(self):
|
||||||
|
supervise.bot_bottle_root = self._orig_root # type: ignore[assignment]
|
||||||
|
self._tmp.cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
class TestCmdDashboardErrorPaths(_FakeHomeMixin, unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self._setup_fake_home()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self._teardown_fake_home()
|
||||||
|
|
||||||
|
def test_keyboard_interrupt_returns_130(self):
|
||||||
|
with mock.patch.object(
|
||||||
|
dashboard.curses, "wrapper", side_effect=KeyboardInterrupt
|
||||||
|
):
|
||||||
|
self.assertEqual(130, dashboard.cmd_dashboard([]))
|
||||||
|
|
||||||
|
def test_die_resurfaces_message_after_curses(self):
|
||||||
|
buf = io.StringIO()
|
||||||
|
with mock.patch.object(
|
||||||
|
dashboard.curses, "wrapper",
|
||||||
|
side_effect=Die(1, "manifest parse error at line 3"),
|
||||||
|
):
|
||||||
|
with contextlib.redirect_stderr(buf):
|
||||||
|
rc = dashboard.cmd_dashboard([])
|
||||||
|
self.assertEqual(1, rc)
|
||||||
|
self.assertIn("manifest parse error at line 3", buf.getvalue())
|
||||||
|
|
||||||
|
def test_die_without_message_has_fallback(self):
|
||||||
|
buf = io.StringIO()
|
||||||
|
with mock.patch.object(dashboard.curses, "wrapper", side_effect=Die(1)):
|
||||||
|
with contextlib.redirect_stderr(buf):
|
||||||
|
rc = dashboard.cmd_dashboard([])
|
||||||
|
self.assertEqual(1, rc)
|
||||||
|
self.assertIn("fatal error", buf.getvalue())
|
||||||
|
|
||||||
|
def test_unexpected_exception_writes_crash_log(self):
|
||||||
|
buf = io.StringIO()
|
||||||
|
with mock.patch.object(
|
||||||
|
dashboard.curses, "wrapper",
|
||||||
|
side_effect=ValueError("kaboom in render"),
|
||||||
|
):
|
||||||
|
with contextlib.redirect_stderr(buf):
|
||||||
|
rc = dashboard.cmd_dashboard([])
|
||||||
|
self.assertEqual(1, rc)
|
||||||
|
out = buf.getvalue()
|
||||||
|
self.assertIn("dashboard crashed: ValueError: kaboom in render", out)
|
||||||
|
self.assertIn("full traceback written to", out)
|
||||||
|
log_path = self._root / "logs" / "dashboard-crash.log"
|
||||||
|
self.assertTrue(log_path.exists())
|
||||||
|
content = log_path.read_text()
|
||||||
|
self.assertIn("kaboom in render", content)
|
||||||
|
self.assertIn("Traceback (most recent call last)", content)
|
||||||
|
|
||||||
|
|
||||||
|
class TestWriteCrashLog(_FakeHomeMixin, unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self._setup_fake_home()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self._teardown_fake_home()
|
||||||
|
|
||||||
|
def test_appends_traceback_with_header(self):
|
||||||
|
try:
|
||||||
|
raise RuntimeError("explode")
|
||||||
|
except RuntimeError as e:
|
||||||
|
path = dashboard._write_crash_log(e)
|
||||||
|
self.assertEqual(self._root / "logs" / "dashboard-crash.log", path)
|
||||||
|
text = path.read_text()
|
||||||
|
self.assertIn("=== dashboard crash", text)
|
||||||
|
self.assertIn("RuntimeError: explode", text)
|
||||||
|
|
||||||
|
def test_falls_back_to_tempfile_when_home_unwritable(self):
|
||||||
|
# bot_bottle_root points at a *file*, so mkdir under it raises
|
||||||
|
# OSError and the helper must fall back to a tempfile.
|
||||||
|
bad = Path(self._tmp.name) / "not-a-dir"
|
||||||
|
bad.write_text("x")
|
||||||
|
supervise.bot_bottle_root = lambda: bad # type: ignore[assignment]
|
||||||
|
try:
|
||||||
|
raise RuntimeError("explode2")
|
||||||
|
except RuntimeError as e:
|
||||||
|
path = dashboard._write_crash_log(e)
|
||||||
|
self.assertTrue(path.exists())
|
||||||
|
self.assertIn("explode2", path.read_text())
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
"""Unit: Docker launch teardown warning on ExitStack failure (issue #156).
|
||||||
|
|
||||||
|
When a callback registered in the ExitStack raises during teardown,
|
||||||
|
the teardown function must emit a WARNING-level message that includes
|
||||||
|
the container name and operation type, rather than silently discarding
|
||||||
|
the exception.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import io
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from bot_bottle.agent_provider import AgentProvisionPlan
|
||||||
|
from bot_bottle.backend import BottleSpec
|
||||||
|
from bot_bottle.backend.docker import launch as launch_mod
|
||||||
|
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||||
|
from bot_bottle.egress import EgressPlan
|
||||||
|
from bot_bottle.git_gate import GitGatePlan
|
||||||
|
from bot_bottle.manifest import Manifest
|
||||||
|
from bot_bottle.pipelock import PipelockProxyPlan
|
||||||
|
from bot_bottle.workspace import workspace_plan
|
||||||
|
|
||||||
|
|
||||||
|
def _manifest() -> Manifest:
|
||||||
|
return Manifest.from_json_obj({
|
||||||
|
"bottles": {"dev": {}},
|
||||||
|
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def _plan(tmp: str) -> DockerBottlePlan:
|
||||||
|
stage = Path(tmp)
|
||||||
|
manifest = _manifest()
|
||||||
|
spec = BottleSpec(
|
||||||
|
manifest=manifest,
|
||||||
|
agent_name="demo",
|
||||||
|
copy_cwd=False,
|
||||||
|
user_cwd=tmp,
|
||||||
|
identity="test-teardown-00001",
|
||||||
|
)
|
||||||
|
return DockerBottlePlan(
|
||||||
|
spec=spec,
|
||||||
|
stage_dir=stage,
|
||||||
|
git_gate_plan=GitGatePlan(
|
||||||
|
slug="test-teardown-00001",
|
||||||
|
entrypoint_script=stage / "entrypoint.sh",
|
||||||
|
hook_script=stage / "hook.sh",
|
||||||
|
access_hook_script=stage / "access-hook.sh",
|
||||||
|
upstreams=(),
|
||||||
|
),
|
||||||
|
egress_plan=EgressPlan(
|
||||||
|
slug="test-teardown-00001",
|
||||||
|
routes_path=stage / "egress.yaml",
|
||||||
|
routes=(),
|
||||||
|
token_env_map={},
|
||||||
|
),
|
||||||
|
supervise_plan=None,
|
||||||
|
agent_provision=AgentProvisionPlan(
|
||||||
|
template="claude",
|
||||||
|
command="claude",
|
||||||
|
prompt_mode="append_file",
|
||||||
|
image="",
|
||||||
|
dockerfile="",
|
||||||
|
guest_env={},
|
||||||
|
),
|
||||||
|
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
|
||||||
|
slug="test-teardown-00001",
|
||||||
|
container_name="bot-bottle-test-teardown-abc",
|
||||||
|
container_name_pinned=False,
|
||||||
|
image="bot-bottle-claude:latest",
|
||||||
|
derived_image="",
|
||||||
|
runtime_image="bot-bottle-claude:latest",
|
||||||
|
dockerfile_path="",
|
||||||
|
env_file=stage / "env",
|
||||||
|
forwarded_env={},
|
||||||
|
prompt_file=stage / "prompt.txt",
|
||||||
|
proxy_plan=PipelockProxyPlan(
|
||||||
|
yaml_path=stage / "pipelock.yaml",
|
||||||
|
slug="test-teardown-00001",
|
||||||
|
),
|
||||||
|
use_runsc=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestTeardownWarning(unittest.TestCase):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
self._tmp = tempfile.mkdtemp(prefix="docker-launch-teardown-test.")
|
||||||
|
|
||||||
|
def tearDown(self) -> None:
|
||||||
|
import shutil
|
||||||
|
shutil.rmtree(self._tmp, ignore_errors=True)
|
||||||
|
|
||||||
|
def test_teardown_failure_emits_warning_with_container_and_operation(self):
|
||||||
|
plan = _plan(self._tmp)
|
||||||
|
buf = io.StringIO()
|
||||||
|
|
||||||
|
with mock.patch.object(launch_mod.docker_mod, "build_image"), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod, "pipelock_tls_init",
|
||||||
|
return_value=(Path("/ca.crt"), Path("/ca.key")),
|
||||||
|
), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod, "egress_tls_init",
|
||||||
|
return_value=(Path("/egress_ca"), Path("/egress_cert")),
|
||||||
|
), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod.network_mod, "network_name_for_slug",
|
||||||
|
return_value="bb-internal-test",
|
||||||
|
), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod.network_mod, "network_egress_name_for_slug",
|
||||||
|
return_value="bb-egress-test",
|
||||||
|
), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod, "bottle_plan_to_compose",
|
||||||
|
return_value={"services": {"agent": {}}},
|
||||||
|
), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod, "write_compose_file",
|
||||||
|
return_value=Path("/tmp/compose.yml"),
|
||||||
|
), \
|
||||||
|
mock.patch.object(launch_mod, "compose_up"), \
|
||||||
|
mock.patch.object(launch_mod, "compose_dump_logs"), \
|
||||||
|
mock.patch.object(
|
||||||
|
launch_mod, "compose_down",
|
||||||
|
side_effect=RuntimeError("network remove failed"),
|
||||||
|
), \
|
||||||
|
contextlib.redirect_stderr(buf):
|
||||||
|
provision = mock.Mock(return_value=None)
|
||||||
|
with launch_mod.launch(plan, provision=provision):
|
||||||
|
pass
|
||||||
|
|
||||||
|
output = buf.getvalue()
|
||||||
|
self.assertIn("bot-bottle: warning:", output)
|
||||||
|
self.assertIn("bot-bottle-test-teardown-abc", output)
|
||||||
|
self.assertIn("compose-down", output)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user