docs/prds/0012-stuck-agent-recovery-flow.md

@@ -6,7 +6,7 @@
 
 ## Summary
 
-When an agent running inside a claude-bottle container gets blocked, it signals via the per-bottle cred-proxy sidecar's `/supervise/notify` endpoint. The supervisor sees the message in a host-side TUI and responds with one of four shapes: a text hint (no infrastructure change, the agent continues); a cred-proxy routes edit (SIGHUP-reload of cred-proxy, agent retries); a pipelock allowlist edit (restart pipelock, agent retries); or an approved manifest diff that triggers a full rebuild of the bottle on the same branch. These map to three categories of stuck — **cred-proxy block**, **pipelock block**, and **capability gap** — described below. The supervisor never opens a live channel into a running bottle; all signal flow goes through the existing internal-network endpoint that cred-proxy already terminates.
+When an agent running inside a claude-bottle container gets blocked, it invokes one of three MCP tool calls — `cred-proxy-block`, `pipelock-block`, or `capability-block` — passing a *proposed* config change (modified `routes.json`, modified pipelock allowlist, or modified agent Dockerfile) plus text describing why the change is justified. The supervisor sees the proposal in a host-side TUI, approves / modifies / rejects it, and the corresponding remediation runs: SIGHUP-reload cred-proxy with the new routes; restart pipelock with the new allowlist; rebuild the bottle from the new Dockerfile on the same branch. The agent's tool call blocks until the operator acts. The supervisor never opens a live channel into a running bottle; all signal flow goes through a per-bottle MCP sidecar on the existing internal network.
 
 ## Problem
 
@@ -14,7 +14,7 @@ Running parallel agents in isolated bottles makes it cheap to spin up work in pa
 
 ## Goals / Success Criteria
 
-A real stuck agent recovers end-to-end in each of the three categories: a **cred-proxy block** is fixed by a `routes edit` + SIGHUP and a "retry now" reply without restarting anything; a **pipelock block** is fixed by an allowlist edit + pipelock restart and a "retry now" reply; a **capability gap** triggers a manifest-diff approval and a bottle rebuild that picks up on the same branch. All three complete without anyone running `docker attach` or opening any live channel into the original container.
+A real stuck agent recovers end-to-end in each of the three categories: a **cred-proxy block** is fixed by the operator approving the agent's proposed `routes.json`, SIGHUP-reloading cred-proxy, and the tool returning "approved, retry now"; a **pipelock block** is fixed by the operator approving the proposed allowlist, restarting pipelock, and the tool returning "approved, retry now"; a **capability block** triggers a bottle rebuild from the proposed Dockerfile, with the replacement agent picking up on the same branch. All three complete without anyone running `docker attach` or opening any live channel into the original container.
 
 ## Non-goals
 
@@ -27,20 +27,19 @@ A real stuck agent recovers end-to-end in each of the three categories: a **cred
 
 ### In scope
 
-- A `/stuck` slash command the agent invokes when blocked. POSTs free-text to cred-proxy's `/supervise/notify` and blocks awaiting a text reply.
-- A `/supervise/notify` endpoint on cred-proxy that persists the agent's message host-side and holds the agent's connection open until the supervisor responds. Wire protocol is text-only: request is the agent's message; response is `{text: "..."}`.
-- SIGHUP-based hot reload of `routes.json` on cred-proxy, so the supervisor can change the agent's credential surface without restarting the proxy or dropping in-flight calls.
-- A TUI dashboard that lists running bottles and pending stuck-notifications. Two verbs:
-  - `r <id> <text>` — reply to a pending stuck message (delivers text to the waiting agent).
-  - `routes edit <bottle>` — open the bottle's `routes.json` in `$EDITOR`, SIGHUP cred-proxy on save. Not gated on a pending message; the supervisor can edit routes anytime.
-- Equivalent support for pipelock: a `pipelock edit <bottle>` TUI verb that opens pipelock's allowlist in `$EDITOR` and restarts pipelock on save. (v1 uses restart, not SIGHUP — see Open questions.)
-- Host-side audit logs at `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log` that record every config edit: timestamp, diff before/after, the operator's reply text if the edit was tied to a `/stuck` reply. Records config shape, never secret values.
-- A rebuild path for the **capability gap** case where the bottle's *manifest* (not just routes or pipelock allowlist) must change. Orchestrator tears down the bottle, applies the approved manifest diff, and starts a replacement bottle on the same branch.
+- A per-bottle MCP sidecar on the bottle's internal network that exposes three tools to the agent: `cred-proxy-block`, `pipelock-block`, `capability-block`. Each tool blocks until the operator acts.
+- Tool input schemas: each tool takes the full proposed file (a new `routes.json`, a new pipelock allowlist, or a new Dockerfile, respectively) plus a `justification` text field. Tool output: `{status: "approved"|"modified"|"rejected", notes: "..."}`.
+- A read-only mount at `/etc/claude-bottle/current-config/` in the agent container that exposes the current `routes.json`, pipelock allowlist, and Dockerfile, so the agent can read them before composing a proposed change.
+- SIGHUP-based hot reload of `routes.json` on cred-proxy, so an approved **cred-proxy block** proposal takes effect without restarting the proxy or dropping in-flight calls.
+- A clean restart path on pipelock that picks up a new allowlist on container restart; v1 does not ship SIGHUP reload for pipelock (see Open questions).
+- A TUI dashboard that lists running bottles and pending tool-call proposals. For each pending proposal: shows the diff (current → proposed) and the agent's justification. Operator actions: approve, modify-then-approve (edit the proposed diff before applying), reject with reason. Also exposes proactive operator-initiated edits: `routes edit <bottle>`, `pipelock edit <bottle>` — useful when the operator wants to change config without waiting for an agent prompt.
+- Host-side audit logs at `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log`. Each entry: timestamp, diff before/after, the agent's justification text (if the edit came from a tool call), and the operator's action (approve / modify / reject) with notes. Records config shape, never secret values.
+- A rebuild path for the **capability block** case where the agent Dockerfile must change. Orchestrator tears down the bottle, applies the approved Dockerfile, and starts a replacement bottle on the same branch.
 - A state-preservation helper for the rebuild path: working tree push is mandatory; transcript / reasoning context is best-effort.
 
 ### Out of scope
 
-- A tool-denial hook that auto-detects "stuck" without the agent's involvement. Deferred to a follow-up; v1 is opt-in via the slash command.
+- A tool-denial hook that auto-detects "stuck" without the agent's involvement. Deferred to a follow-up; v1 is opt-in via the agent calling one of the three MCP tools.
 - A web dashboard. TUI only in v1.
 - Live channel into running containers (see Non-goals).
 - Agent-to-agent communication (see Non-goals).
@@ -50,57 +49,63 @@ A real stuck agent recovers end-to-end in each of the three categories: a **cred
 
 ### Stuck categories
 
-Three named categories, ordered by remediation cost:
+Three named categories, each with its own MCP tool. Ordered by remediation cost:
 
-- **cred-proxy block.** The agent's request was refused by cred-proxy — missing route, expired token, wrong scope. The bottle is otherwise healthy. *Remediation:* operator runs `routes edit <bottle>`, edits `routes.json`, saves. cred-proxy SIGHUP-reloads; in-flight connections are not dropped. Operator replies to the `/stuck` message with a "retry now" hint. The agent retries against the (now-reloaded) cred-proxy and proceeds.
-- **pipelock block.** The agent's outbound request was refused by pipelock — host not in the allowlist, protocol not permitted, etc. The bottle is otherwise healthy, but the egress perimeter is wrong. *Remediation:* operator runs `pipelock edit <bottle>`, edits the allowlist, saves. pipelock restarts; the agent's in-flight outbound calls may drop and need retry. Operator replies to the `/stuck` message with a "retry now" hint. (v1 uses restart; SIGHUP reload for pipelock is an Open question.)
-- **capability gap.** The bottle is missing something the agent needs that lives in the manifest itself — a tool, a skill, a permission grant, an env var. Routes and pipelock are correct; the agent container just doesn't have the capability. *Remediation:* operator approves a manifest diff in the TUI. The rebuild orchestrator tears down the bottle, applies the diff, and starts a replacement bottle on the same branch via the state-preservation helper. The replacement agent picks up where the original was, now with the missing capability.
+- **cred-proxy block.** Tool: `cred-proxy-block`. The agent's request was refused by cred-proxy — missing route, expired token, wrong scope. The agent reads the current `routes.json` from `/etc/claude-bottle/current-config/`, composes a modified version, and calls the tool with `{routes: <new file>, justification: "..."}`. The operator reviews the diff in the TUI; on approval, the supervisor writes the new `routes.json` and cred-proxy SIGHUP-reloads. In-flight connections are not dropped. The tool returns `{status: "approved", notes: "..."}` and the agent retries.
+- **pipelock block.** Tool: `pipelock-block`. The agent's outbound request was refused by pipelock — host not in the allowlist, protocol not permitted. The agent reads the current allowlist, composes a modified version, and calls the tool with `{allowlist: <new file>, justification: "..."}`. On approval, the supervisor writes the new allowlist and restarts pipelock; in-flight outbound calls may drop and rely on retry. The tool returns the same approve/reject shape.
+- **capability block.** Tool: `capability-block`. The bottle is missing a tool, skill, permission, or env var the agent needs — something that lives in the agent Dockerfile rather than in routes or the pipelock allowlist. The agent reads the current Dockerfile, composes a modified version, and calls the tool with `{dockerfile: <new file>, justification: "..."}`. On approval, the rebuild orchestrator tears down the bottle, builds from the new Dockerfile, and starts a replacement bottle on the same branch via the state-preservation helper. Because the current agent is about to be replaced, the tool's return is best-effort — the replacement agent inherits the approval record via the preserved transcript.
 
-The wire protocol does not change between categories: the agent POSTs free text to `/supervise/notify` and receives `{text: "..."}`. The category is the operator's mental model for triage, not a field on the request. The agent does not need to know which category its message will fall into.
+The three tools are dispatched by name, so the operator's TUI knows which remediation engine to wire to which proposal. The agent must choose the right tool for what failed: a 403 from a credentialed request is a `cred-proxy-block`; a connection refused at the egress is a `pipelock-block`; a "command not found" or missing-skill error is a `capability-block`.
 
 ### New services / components
 
-- **`/stuck` slash command.** Shipped as a skill mounted into bottles. POSTs the agent's free-text message to cred-proxy's `/supervise/notify` and blocks awaiting a text reply. Reply text is handed back to the agent verbatim — the agent doesn't need to know whether the supervisor edited routes, opened an editor, or did anything else before composing the reply.
-- **cred-proxy `/supervise/notify` endpoint.** Receives the agent's message, persists it to a host-mounted queue, and holds the agent's connection open until the supervisor responds. The wire protocol is text-only in both directions; the supervisor's side-effects (routes edit, manifest diff, no-op) are invisible to the agent.
-- **cred-proxy SIGHUP reload.** New behavior on the existing process: SIGHUP re-reads `routes.json` without dropping connections or breaking in-flight calls. ~30 lines added to the server. Used by the **cred-proxy block** category.
-- **pipelock edit + restart.** v1 ships restart-based reload for pipelock: on `pipelock edit <bottle>` save, the supervisor writes the new allowlist and restarts the pipelock container. The agent's in-flight outbound calls drop and rely on retry. Used by the **pipelock block** category.
-- **TUI dashboard.** A `claude-bottle dashboard` (or similarly named) command. Lists running bottles, surfaces pending stuck-notifications, exposes the `r <id> <text>`, `routes edit <bottle>`, and `pipelock edit <bottle>` verbs, and (for the **capability gap** category) shows proposed manifest diffs with approve/reject input. Targets stdlib only; a TUI library is added only if the experience truly demands it.
-- **Config-edit audit logs.** `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log`. Every edit appends: timestamp, diff before/after, operator's reply text if tied to a `/stuck` reply. Records what the bottle's credential surface and egress perimeter looked like at time T without storing secret values.
-- **Rebuild orchestrator (capability-gap path).** Used when the manifest itself must change, not just routes or the pipelock allowlist. On approval, tears down the existing bottle, applies the approved manifest diff, snapshots state via the state-preservation helper, and starts a fresh bottle on the same branch.
-- **State-preservation helper.** Mandatory: ensures the working tree is pushed before teardown. Best-effort: carries forward the agent's transcript / reasoning context into the replacement container so the new agent starts warm rather than cold.
+- **MCP sidecar.** New per-bottle container on the bottle's internal network. Exposes the three tools (`cred-proxy-block`, `pipelock-block`, `capability-block`) to the agent. On a tool call: validates the proposed file syntactically (valid JSON for `routes.json`, parseable Dockerfile, etc.), persists the proposal to a host-mounted queue, and holds the tool-call connection open until the supervisor acts. On a response from the supervisor, returns `{status, notes}` to the agent. Whether this lives as its own container or as a mode of cred-proxy is an Open question; v1 plan is its own container to keep cred-proxy focused on credentials.
+- **`cred-proxy-block` MCP tool.** Input: `{routes: <full file>, justification: <text>}`. Output: `{status: "approved"|"modified"|"rejected", notes: <text>}`. On approval, the supervisor writes the new `routes.json` and SIGHUPs cred-proxy.
+- **`pipelock-block` MCP tool.** Input: `{allowlist: <full file>, justification: <text>}`. Output: same approve/reject shape. On approval, the supervisor writes the new allowlist and restarts pipelock.
+- **`capability-block` MCP tool.** Input: `{dockerfile: <full file>, justification: <text>}`. Output: best-effort approval acknowledgement (the agent is about to be torn down). On approval, the rebuild orchestrator runs.
+- **Read-only current-config mount.** `/etc/claude-bottle/current-config/` in the agent container exposes `routes.json`, the pipelock allowlist, and the agent Dockerfile from the host. Read-only — the agent proposes changes via the tool call, never by writing the file directly.
+- **cred-proxy SIGHUP reload.** New behavior on the existing process: SIGHUP re-reads `routes.json` without dropping connections or breaking in-flight calls. ~30 lines added to the server. Used by the **cred-proxy block** path.
+- **pipelock restart path.** v1 uses restart, not SIGHUP: the supervisor writes the new allowlist and restarts the pipelock container. Agent's in-flight outbound calls drop and rely on retry. Used by the **pipelock block** path.
+- **TUI dashboard.** A `claude-bottle dashboard` (or similarly named) command. Lists running bottles and pending tool-call proposals. For each proposal: shows current vs. proposed diff and the agent's justification; operator actions are approve, modify-then-approve, or reject-with-reason. Also exposes proactive operator-initiated edits: `routes edit <bottle>` and `pipelock edit <bottle>` — useful when the operator wants to change config without waiting for an agent prompt. Targets stdlib only; a TUI library is added only if the experience truly demands it.
+- **Config-edit audit logs.** `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log`. Every edit appends: timestamp, diff before/after, the agent's justification (if the edit came from a tool call), and the operator's action with notes. Records config shape, never the secret values themselves.
+- **Rebuild orchestrator (capability-block path).** Used when the Dockerfile must change. On approval, tears down the existing bottle, builds from the new Dockerfile, snapshots state via the state-preservation helper, and starts a fresh bottle on the same branch.
+- **State-preservation helper.** Mandatory: ensures the working tree is pushed before teardown. Best-effort: carries forward the agent's transcript / reasoning context — including the approved `capability-block` proposal — into the replacement container so the new agent starts warm rather than cold.
 
 ### Existing code touched
 
-- **cred-proxy** (PRD 0010) — gains the `/supervise/notify` endpoint, the host-mounted notification queue, and SIGHUP reload of `routes.json`.
+- **cred-proxy** (PRD 0010) — gains SIGHUP reload of `routes.json`. The supervise-notify responsibility moves out of cred-proxy and into the new MCP sidecar, keeping cred-proxy focused on credentials.
 - **pipelock** — gains a clean restart path that picks up the new allowlist on container restart. No code changes likely needed if pipelock already reads its config on startup; the orchestration is supervisor-side.
-- **`cli.py`** — gains the dashboard subcommand (with `r`, `routes edit`, and `pipelock edit` verbs) and the rebuild path.
-- **Bottle lifecycle scripts** — extended for orchestrated teardown + rebuild with state hand-off, distinct from a fresh-spawn.
+- **`cli.py`** — gains the dashboard subcommand (approve/modify/reject pending tool-call proposals; `routes edit` and `pipelock edit` for proactive operator-initiated changes) and the rebuild path.
+- **Bottle lifecycle scripts** — extended to launch the MCP sidecar alongside the other sidecars; mount the read-only current-config directory into the agent container; orchestrate teardown + rebuild with state hand-off.
 - **Bottle manifest schema** — may need to record the originating manifest version / change history per agent run, so the dashboard can show "what changed" rather than "what is."
 
 ### Data model changes
 
-- A per-bottle pending-notification queue: cred-proxy holds the agent's open connection; the queue holds the metadata (id, bottle slug, message body, arrival timestamp) the TUI needs to render the ask.
-- Per-bottle config audit log files at `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log`, append-only.
+- A per-bottle pending-proposal queue: the MCP sidecar holds the agent's open tool-call connection; the queue holds the proposal payload (id, bottle slug, tool name, current file hash, proposed file content, agent's justification text, arrival timestamp) the TUI needs to render the ask.
+- Per-bottle config audit log files at `~/.claude-bottle/audit/cred-proxy-<slug>.log` and `~/.claude-bottle/audit/pipelock-<slug>.log`, append-only. Each entry includes the agent's justification and the operator's action.
 - A per-agent-run record sufficient to map a running bottle back to its PR / branch, so the rebuild orchestrator knows where to post the comment and which branch to resume on.
 
 ### External dependencies
 
 - The Gitea API / `tea` CLI is already in the toolbox (the project is on Gitea); no new auth surface beyond what the orchestrator already needs to read/post on PRs.
 - A TUI library is a *maybe* — only if stdlib can't carry the dashboard experience. Default to no new dependency.
+- An MCP server library / framework. The MCP sidecar implements the MCP protocol so the agent can call the three tools natively. Pick the lightest option that lets the sidecar advertise three tools with structured input/output schemas; do not adopt a heavier MCP framework than the three tools justify.
 
 ## Open questions
 
+- Text-only vs. structured tools. Earlier drafts of this PRD used a text-only protocol (`/supervise/notify` returning `{text}`); this revision uses three structured MCP tools that carry the agent's proposed file. **Structured wins on:** richer triage signal (operator sees the diff up front, not just a description of it), cleaner audit (the agent's proposed shape is captured alongside the operator's action), and the agent does diff-authoring work the operator would otherwise have to do. **Structured costs:** larger wire surface, the agent has to know the file formats (`routes.json` schema, Dockerfile syntax, pipelock allowlist format), miscategorization is possible (e.g. a 403 the agent reads as a `cred-proxy-block` might actually be a pipelock issue at a different layer). **Text-only wins on:** smallest possible protocol, no schema burden on the agent, easy to extend (every new category is just another reason in prose). **Text-only costs:** operator does all the diff authoring, audit log loses the agent's proposed shape, no opportunity for the agent's understanding of the fix to be inspected. Worth re-litigating if the MCP sidecar grows complex relative to the value it produces.
+- MCP sidecar placement. v1 plan is its own container. Alternative is folding the supervise plane back into cred-proxy as a second tool surface. Own container keeps separation clean; folded saves one sidecar per bottle. Worth deciding once the sidecar's actual line count is known.
+- `capability-block` return semantics. The current agent is torn down on approval, so the tool's return value never reaches it. Options: (a) fire-and-forget, the tool returns immediately with "queued" and the agent halts; (b) block the tool, let the rebuild orchestrator's teardown kill the connection, replacement agent gets the approval record via state-preservation; (c) the tool blocks, returns "approved" right before teardown, the agent has milliseconds to log it. (b) seems cleanest but is worth confirming during implementation.
 - SIGHUP race window. An agent that retries within msec of the SIGHUP may hit old routes once before the reload completes, fail, and retry against the new routes. Assumption is that normal HTTP retry semantics absorb this; worth confirming under real usage rather than designing around it preemptively.
 - SIGHUP reload for pipelock. v1 ships restart-based reload, which drops in-flight outbound calls. Should pipelock gain SIGHUP support so **pipelock block** is as cheap as **cred-proxy block**? Depends on how often the operator edits the allowlist mid-task and how disruptive a pipelock bounce actually is.
-- Multiple pending notifications from the same bottle. If the agent calls `/stuck` again before the prior message is answered, what does the queue do — replace, append, or refuse? Append feels safest; replace is wrong (loses context); refuse forces the agent to handle a new error mode.
-- Verb naming under load. `r <id> <text>` optimizes for muscle memory mid-incident; `reply <id> <text>` reads better cold. Worth picking once and committing.
+- Multiple pending proposals from the same bottle. If the agent calls a second tool before the first is answered, what does the queue do — replace, append, or refuse? Append feels safest; replace is wrong (loses context); refuse forces the agent to handle a new error mode. Also: can different tools from the same bottle be pending simultaneously (e.g. a `cred-proxy-block` and a `pipelock-block` in flight at once)?
+- Proposal validation strictness. The MCP sidecar validates the proposed file syntactically. Should it also do a deeper check — e.g. does the proposed `routes.json` introduce a route the operator already rejected this session? Probably no for v1; the operator is the gate.
 - Best-effort transcript preservation on the rebuild path. Mount the agent's state directory, snapshot on teardown, remount in the replacement? How much fidelity is "good enough" for the new agent to pick up?
-- Tool-denial auto-detection. Should v1 also ship a tool-denial hook that auto-invokes `/stuck` without the agent's involvement, or strictly the agent-initiated form? Currently deferred; line worth confirming during implementation.
-- Rejection semantics on the rebuild path. Does the agent receive a `/stuck` reply explaining the rejection, or does the bottle just stay torn down?
+- Tool-denial auto-detection. Should v1 also ship a denial hook that auto-invokes one of the three tools without the agent's reasoning step, or strictly the agent-initiated form? Currently deferred; agent-initiated is safer (the agent has the most context about *why* it needed the call that was denied).
 - Bottle → PR/branch mapping. Recorded at bottle-spawn time, derived from the working tree, or specified in the manifest?
-- How does the flow handle one-off exceptions to gitlock / pipelock denials — e.g. a commit that includes docs with intentionally-bogus tokens that the secret scanner correctly flags? The shape (agent blocked → `/stuck` → operator decides → reply) is the same, but the *resolution* differs: a per-operation override or a scoped allowlist entry, not a routes edit or a manifest change. Does the operator express the exception by commit SHA, by content hash, or by a narrow allowlist rule? Either way, the approval must be auditable so a future reader can see what was waived and why. See `docs/research/git-gate-commit-approval.md` for a survey of gitleaks's native allowlist primitives and a recommendation.
+- How does the flow handle one-off exceptions to gitlock / pipelock denials — e.g. a commit that includes docs with intentionally-bogus tokens that the secret scanner correctly flags? The shape (agent blocked → tool call → operator decides → result) is the same, but the *resolution* differs: a per-operation override or a scoped allowlist entry, not a routes edit, allowlist edit, or Dockerfile change. Is this a fourth tool (`exception-block`?), or does it fold into `pipelock-block` with a scoped one-shot allowlist entry? Either way, the approval must be auditable so a future reader can see what was waived and why. See `docs/research/git-gate-commit-approval.md` for a survey of gitleaks's native allowlist primitives and a recommendation.
 
 ## References
 
-- PRD 0010 — cred-proxy (the endpoint extended to carry stuck-requests).
+- PRD 0010 — cred-proxy (gains SIGHUP reload of `routes.json`; the supervise plane lives in a separate MCP sidecar).
 - `CLAUDE.md` — project non-goal on agent-to-agent communication; this PRD stays on the human→agent side of that line.