Survey the three realistic ways to surface a claude-code session
inside the dashboard TUI:
1. Handoff — drop curses, foreground claude, restore on exit
(the existing `e`/`p` pattern, extended). Minimal code,
side-by-time rather than side-by-side.
2. Embedded emulator — own a PTY, parse claude-code's ANSI
stream via `pyte`, paint it into a curses pane. Real
"pane in the dashboard" but a six-week build with one new
dep and several integration trap-doors (alt-screen, resize,
input routing, multi-PTY state).
3. External multiplexer — delegate pane creation to tmux /
iTerm / wezterm when detected. Tiny code, but splits the
operator's mental model and gives up layout control.
Recommendation: ship Option 1 first; defer Option 2 to "only if
Option 1 is observably insufficient"; treat Option 3 as a
niche augmentation for power users.
Calls out four followups worth verifying before committing
(PTY behavior at small sizes, attach-to-existing-exec, SIGWINCH
handling, `-it` vs `-i` for the embedded path).
PRD 0018 chunk 3's atomicity fix used write-temp-then-rename to
update bind-mounted config files. POSIX rename atomically swaps
the inode at the host path — but Docker single-file bind mounts
on Linux pin the source inode at mount time, so post-rename the
container's mount points at the now-orphaned old inode and never
sees the new content. The egress sidecar's SIGHUP-driven reload
re-reads the same stale file → "egress route updates aren't
updatable via the supervisor anymore".
Switch egress_apply + pipelock_apply to write in place (same
inode, truncated + rewritten). Lose file-level POSIX atomicity,
but:
- egress: SIGHUP fires only AFTER the write returns; the
addon's `load_routes` raises `ValueError` on a partial read
and keeps the previous in-memory routes, so the in-process
race window (already narrow) is non-disruptive.
- pipelock: applies via `docker restart` rather than SIGHUP;
restart serializes after the host write completes, so the
container reads the fully-written file on next boot.
macOS Docker Desktop's file-sharing layer (virtiofs / osxfs)
silently re-resolves the path on rename, which is why this bug
didn't surface in dev tests on macOS. Linux native Docker is
the strict reading; the fix works on both.
`egress_render_routes` now emits hand-rolled YAML in the same style
as `pipelock_render_yaml`. The egress addon parses it via
`yaml_subset.parse_yaml_subset` — the same parser the manifest
loader + pipelock_apply use.
Why bother: routes.yaml is bind-mounted into the egress sidecar
AND surfaced to operators through `routes edit` (PRD 0019). JSON-
in-yml renders ugly in $EDITOR and signals "this is data" rather
than "this is config you can read at a glance". Real YAML reads
cleanly.
Mechanics:
- `yaml_subset.py` drops its `claude_bottle.log` dependency.
Errors now raise `YamlSubsetError` (a `ValueError`); the
manifest loader + pipelock_apply catch it at the boundary
and forward to `die` / `PipelockApplyError` so callers see
the same behavior they did before.
- `Dockerfile.egress` adds one COPY line for `yaml_subset.py`
so it sits flat in `/app/` next to the addon. The addon
uses an absolute-import-with-fallback shim so the same file
works inside the container AND from the host's unit tests.
- `egress_apply._merge_single_route` round-trips current
routes.yaml through `parse_yaml_subset` + a new
`_render_routes_payload` helper instead of `json.loads` +
`json.dumps`.
End-to-end: rebuilt the egress image, ran `./cli.py start` to a
full bring-up, confirmed the addon's boot log shows `egress:
loaded 9 route(s)` — i.e., the YAML parses inside the container.
453 unit + 3 integration tests pass.
PRD 0019 chunk 4 (final). The `e` (routes edit) and `p` (pipelock
edit) keys now require an agent selection in the agents pane.
Pressing them with the proposals pane focused, with no active
agents, or with an out-of-range selection is a no-op with a
status hint ("no agent selected; Tab into the agents pane first").
The discover-and-prompt scaffolding inside
`_operator_edit_routes_flow` / `_operator_edit_allowlist_flow` /
`_operator_edit_flow` is gone. The flows now take an `ActiveAgent`
+ required-service name; they refuse with a clear message when
the bottle lacks the requested sidecar (e.g., `routes edit`
against a bottle with no `bottle.egress.routes` declared). The
`discover_egress_slugs` + `discover_pipelock_slugs` +
`_discover_active_with_service` helpers come out — they had no
remaining callers.
Footer now reads `[e/p] edit selected agent`.
PRD 0019 chunk 3. The TUI now has two focusable panes — proposals
and agents — and `Tab` toggles which one the `j/k`/arrow keys
move through.
Each pane keeps its own selection index. Switching panes doesn't
lose the position in the other; the cursor (`>` + reverse-video
row) appears only in the focused pane. The label line on each
pane shows "(focused)" when active.
Footer reshuffled: `[Tab] switch pane [j/k] move [Enter] view
[a/m/r] proposal [e/p] edit [q] quit`. When the agents pane is
focused and there's no status message to display, the idle
status line surfaces the currently-selected agent (or "[no
active agents]" / "[no agent selected]" fallbacks) so the
operator knows what an agent-scoped edit verb will target after
chunk 4 wires them up.
Proposal action keys (a/m/r/Enter) are gated on the proposals
pane being focused — pressing them with the agents pane focused
is a no-op. e/p still use the global discover-and-prompt flow
for one more chunk; chunk 4 swaps them to read the agents-pane
selection.
PRD 0019 chunk 2. The TUI's main render now draws two panes:
proposals on top (existing), active agents on the bottom (new).
Header counts both totals. The agents pane refreshes on the
same 1s tick — agents starting/stopping reflect without
operator action.
Each agent row shows slug, agent name, started-time (HH:MM:SS
of the metadata.json timestamp), and the bracketed list of
sidecars currently up. The `agent` service is filtered out of
the displayed list — it's always present so it'd be noise; the
sidecars are the differentiator. A bottle whose only running
service is `agent` (sidecars still warming up) renders as
`(starting)`.
No selection model yet — that's chunk 3. The cursor stays in
the proposals pane; `j/k`/arrow nav and the proposal action
keys are unchanged.
PRD 0019 chunk 1. New `discover_active_agents()` in dashboard.py
returns one `ActiveAgent(slug, agent_name, started_at, services)`
per currently-running compose project:
- Slugs come from `list_active_slugs()` (chunk-5 shared helper).
- The service set per project comes from ONE label-filtered
`docker ps` call (PRD open question #1: avoids N per-bottle
`compose ps` invocations on each 1s refresh tick).
- agent_name + started_at come from each bottle's
metadata.json; "?" / "" fallbacks when the file is missing
so the row renders rather than vanishes.
Not wired into the TUI yet — chunk 2 renders the agents pane.
The parser (`_parse_services_by_project`) is split out as a pure
function so the conditional-input shape can be unit-tested
without docker.
When no agent is selected, `e` / `p` do nothing (status line
shows "no agent selected") rather than falling back to today's
global discover-and-prompt. The discover-and-prompt scaffolding
in `_operator_edit_routes_flow` / `_operator_edit_allowlist_flow`
comes out entirely — selection in the agents pane is now the
only way to scope an edit. Old open-question #4 (single-bottle
shortcut behavior in proposals-pane mode) is moot and removed.
Draft a PRD that adds an "active agents" pane to the dashboard
TUI (below the existing proposals pane) and reshapes the operator
`routes edit` (e) / `pipelock edit` (p) verbs to be agent-scoped
when the cursor is in the agents pane — no more global discover
+ disambiguation prompt on every press. Tab toggles which pane
nav keys move through.
Sized into 4 chunks (discovery helper → render pane → selection
state → agent-scoped verbs). Six open questions called out, the
biggest being whether per-bottle `compose ps` on every 1s tick
scales for hosts with many bottles (answer leans toward one
label-filtered `docker ps`).
PRD 0018 chunk 5. The dashboard's operator-edit verbs
(`routes edit`, `pipelock edit`) enumerated running sidecars
via `docker ps --filter name=...` prefix scans. Switch to
`docker compose ls`-based discovery so the dashboard, cleanup
CLI, and launch step all agree on what's running.
Mechanics:
- `claude_bottle/backend/docker/compose.py` grows three shared
helpers: `list_compose_projects` (the JSON parse moved out
of cleanup), `slug_from_compose_project` (inverse of
`compose_project_name`), and `list_active_slugs` (sugar over
the first two for the common "what's running?" question).
- cleanup.py drops its private `_list_compose_projects` +
`_PROJECT_PREFIX` in favor of the shared ones; `list_active`
simplifies (one compose-ls call, not two).
- dashboard.py's `_discover_sidecar_slugs` becomes
`_discover_active_with_service`: cross-references the active
slug list with a label-filtered `docker ps` so only bottles
whose given service container is actually up surface in the
edit menu. Bottles without an egress sidecar (no
bottle.egress.routes) no longer appear for `routes edit`.
3 new unit tests cover the slug ↔ compose-project naming
contract; manual probe with a fake compose project confirms
both `discover_egress_slugs` and `discover_pipelock_slugs`
return the expected slug.
PRD 0018 chunk 4. `claude-bottle cleanup` now derives its work
from `docker compose ls --all --format json`, filtered to projects
whose name starts with `claude-bottle-`. Per project: one `compose
down --volumes` removes the containers + the compose-managed
networks atomically.
The plan also enumerates three fallback buckets:
- Stray containers — `claude-bottle-*` containers with no
`com.docker.compose.project` label (left over from pre-compose
code paths). Cleared via `docker rm -f`.
- Stray networks — `claude-bottle-*` networks with no compose
project label. Cleared via `docker network rm`.
- Orphan state dirs — per-bottle `~/.claude-bottle/state/<id>/`
dirs with no live project AND no `.preserve` marker. The
`.preserve` marker (capability-block or auto-preserve-on-crash)
explicitly opts-out of reaping; manual `rm -rf` is the only
path for preserved state.
cli/cleanup.py collapses to a single y/N prompt — backend.prepare_cleanup
returns everything in one plan, backend.cleanup processes everything,
no more double-prompt for state. The CLI-side state-dir enumeration
+ `_state_summary` flags from PR #25 are gone; the backend's
orphan-detection rules subsume them.
PRD 0018 chunk 4 spike: empirically verified that pipelock's SSRF
guard checks proxied-request destinations (e.g. api.anthropic.com →
public IP) and not source IPs of incoming connections. The
bottle's own internal CIDR was being added to ssrf.ip_allowlist
defensively, but that defense isn't load-bearing — direct pipelock
probe (`curl --proxy http://pipelockhttps://api.anthropic.com/`)
returns 404 from upstream rather than blocking on SSRF.
So:
- Networks become compose-managed (`internal: true` on the
internal network; the egress one is a normal user-defined
bridge). Compose creates + removes them via up/down.
- launch.py drops the `docker network create` + `network_inspect_cidr`
+ pipelock yaml re-render dance.
- The pre-create/external scaffolding from chunk 3 goes with it.
End-to-end `./cli.py start` still works; cleanup leaves no
orphans. If real-world use surfaces an SSRF block we hadn't
predicted, the allowlist can come back via subnet-pinning rather
than pre-create.
PRD 0018 chunk 3. Each instance is now one `docker compose` project:
- launch.py renders the compose spec via chunk-1's
bottle_plan_to_compose, writes it to state/<slug>/docker-compose.yml,
`docker compose up -d`s, and (on teardown) dumps
`docker compose logs --no-color --timestamps` to
state/<slug>/compose.log before `docker compose down`.
- Networks are pre-created (`docker network create --internal` +
user-defined bridge) so pipelock yaml can know the internal CIDR
before compose-up. Compose references them with `external: true`;
the launch step's ExitStack still owns network removal.
- Agent still runs `sleep infinity`; claude reaches it via
`docker exec -it` exactly like before (per the PRD's resolved
TTY question).
- metadata.json grows a `compose_project` field so dashboard /
cleanup tooling can derive compose invocations without
re-deriving the slug.
Security follow-ups from chunk-2 review:
(b) CA private keys: pipelock + egress ca-key.pem land at 0o600
explicitly. The mitmproxy cert+key concat stays 0o644 because
the egress container's uid-1000 user reads it through the
bind mount; parent dir at 0o700 still restricts host-side
reach.
(c) Apply atomicity: egress_apply + pipelock_apply switch from
`docker cp` to host-side write-temp-then-rename on the
bind-mount source. POSIX rename is atomic on the same
filesystem, so a sidecar SIGHUP racing the apply can't see
a half-written routes.yaml / pipelock.yaml.
Per-sidecar Docker{Sidecar}.start/stop methods stay in place — the
integration test suite drives them directly to validate each image
in isolation, which is still useful. launch.py no longer calls
them; a follow-up chunk can prune if the integration tests move to
the compose lifecycle.
git-gate entrypoint's chmod 600 on the keyfile + known_hosts now
tolerates EROFS (`|| true`) — the host SSH key is already 0600
(SSH refuses to load otherwise), so the inside-container chmod
was already a no-op in the docker-cp path and now just needs to
not error on the read-only bind mount.
422 unit tests pass; supervise integration test passes; end-to-end
`./cli.py start implementer` brings up the project, attaches,
captures full merged logs on teardown, and reaps all containers +
networks.
PRD 0018 chunk 2. Each sidecar's prepare-time output (pipelock yaml +
CAs, egress routes.yaml + CAs, git-gate entrypoint + hooks, supervise
current-config, agent env + prompt) now lands in
~/.claude-bottle/state/<slug>/<service>/ instead of an ephemeral
mktemp dir. The state subdirs become the stable bind-mount sources
that chunk 3's docker compose project will reference.
The SDK launch path is unchanged — `docker cp` still copies from the
plan-held paths into containers, just from new locations. start.py's
session-end cleanup is now in `finally`, which also reaps state dirs
left behind by dry-run / preflight-N / prepare-exception paths
(previously only the post-launch path settled state).
PRD 0018 chunk 1. New module `claude_bottle/backend/docker/compose.py`
exposing `bottle_plan_to_compose(plan) -> dict` — a pure function that
translates a fully-resolved DockerBottlePlan into a Compose v2 spec.
Not wired in yet. Tests cover the conditional-service matrix (git
on/off × egress on/off × supervise on/off) plus per-service shape
(images vs builds, network aliases, bind mounts, env vars, depends_on).
Draft a PRD that replaces the chain of per-sidecar docker SDK calls
in `claude-bottle start` with a single `docker compose` project per
instance. Each `state/<slug>/` dir gets a self-describing set of
artifacts: metadata.json, docker-compose.yml, compose.log, and the
existing transcript/ + live-config/.
The manifest key is `egress:` now; finish the rename so the rest of
the codebase matches. Files (Dockerfile.egress, claude_bottle/egress.py
etc.), classes (Egress, EgressConfig, EgressRoute, EgressPlan,
DockerEgress), constants (EGRESS_HOSTNAME, EGRESS_ROUTES, ...),
container name prefix (claude-bottle-egress-*), docker network alias
(egress), the introspection host (_egress.local), the MCP tool IDs
(egress-block, list-egress-routes), and the preflight label all drop
the `-proxy` suffix.
Now that `bottle.egress` (the old allowlist/dlp_action block) is
gone, the longer `egress_proxy:` disambiguator isn't needed. The
manifest field reads more naturally as just `egress:` with the
same nested `routes: [...]` shape.
Renamed:
- Manifest YAML key: `egress_proxy:` → `egress:`
- Bottle dataclass attr: `bottle.egress_proxy` → `bottle.egress`
- `_BOTTLE_KEYS` entry, schema docstring, and all
user-facing error message labels (`egress.routes[N]`,
`egress has unknown key …`, etc.).
Kept (these refer to the egress-proxy SIDECAR, not the manifest
field):
- File names: `egress_proxy.py`, `egress_proxy_apply.py`,
`egress_proxy_addon.py`, `egress_proxy_addon_core.py`.
- Class names: `EgressProxyConfig`, `EgressProxyRoute`,
`EgressProxyPlan`, `EgressProxy`, `DockerEgressProxy`.
- Helper names: `egress_proxy_manifest_routes`,
`egress_proxy_routes_for_bottle`,
`egress_proxy_token_env_map`, etc.
- Constants: `EGRESS_PROXY_HOSTNAME`, `EGRESS_PROXY_ROLES`,
`EGRESS_PROXY_AUTH_SCHEMES`, `EGRESS_PROXY_FORWARD_PROXY`,
`EGRESS_PROXY_INTROSPECT_URL`, `EGRESS_PROXY_PORT`, etc.
- Container name prefix `claude-bottle-egress-proxy-*`, the
`egress-proxy` docker network alias, the
`egress-proxy-block` + `list-egress-proxy-routes` MCP tool
IDs, the `egress-proxy` audit-log component label.
Local bottle migrated (`~/.claude-bottle/bottles/dev.md` already
updated). The legacy `egress_proxy` key isn't surfaced anywhere
anymore; the generic unknown-key validator catches typos with a
"did you mean: egress, env, git, supervise" hint.
409 unit + integration tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Goal: one allowlist surface (egress_proxy.routes), no second
free-form `egress:` knob. Anything that used to live there now
goes in `egress_proxy.routes` as a bare-pass entry
(`- host: <name>`).
Removed:
- `BottleEgress` dataclass + DLP_ACTIONS constant + bottle.egress
field on `Bottle`.
- `pipelock_bottle_allowlist` helper.
- `pipelock_allowlist_summary` helper (the compact preflight
summary stopped using it after PR #31).
- `allowlist_summary` field on `DockerBottlePlan`.
- `bottle.egress.allowlist` folding in
`egress_proxy_routes_for_bottle` — only DEFAULT_ALLOWLIST
auto-folds now.
- The two-branch logic in `pipelock_effective_allowlist`
(egress-proxy-present vs not) — pipelock now just mirrors
`egress_proxy_routes_for_bottle` unconditionally.
Hard-coded:
- `request_body_scanning.action = "block"` in
`pipelock_build_config` (was driven by
`bottle.egress.dlp_action`). The previous default was already
"block" — the knob to switch to "warn" was a foot-gun in a
sandboxed agent context, so it's gone.
Tests:
- `test_pipelock_allowlist.py` rewritten to assert the
mirrored-from-egress-proxy semantics directly.
- `test_manifest_md_load.py`, `test_pipelock_yaml.py`,
`test_egress_proxy.py` fixtures migrated to put hosts in
`egress_proxy.routes` instead of `egress.allowlist`.
Local bottle migrated too: `~/.claude-bottle/bottles/dev.md`
loses the `egress: { allowlist: [example.com] }` block, picks up
a bare-pass `- host: example.com` route.
409 unit + integration tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Companion to the compact preflight in #31 — the JSON format was
the structured alternative to the verbose text summary. With the
new compact text already on screen, no consumer was using the
JSON shape, and the abstract `BottlePlan.to_dict` was the
biggest piece of API surface no one is implementing against.
Removed:
- `--format` CLI flag from `start` and `resume`.
- `output_format` kwarg from `_launch_bottle`.
- `BottlePlan.to_dict` abstract method.
- `DockerBottlePlan.to_dict` (60-line dict builder).
- The `_PlanView` dataclass — `print` was the only remaining
caller, so the env-name computation is inlined.
- `tests/integration/test_dry_run_plan.py` (JSON-shape
integration test).
- `tests/unit/test_cli_start_format.py` (flag-conflict unit).
Plan-introspection is still possible by reading the
`DockerBottlePlan` dataclass directly — fields like `image`,
`container_name`, `stage_dir`, `use_runsc` are all there. Tooling
that needs a stable wire shape can JSON-serialize the dataclass
themselves.
411 unit + integration tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Trim the y/N preflight to the parts the operator actually scans
before pressing y:
agent
env (one per line)
skills (one per line)
bottle
git gate (one upstream per line)
egress-proxy (one route per line, with [auth:scheme] when set)
Dropped from the display (still on the plan dataclass / json
output for tooling): image, dockerfile, derived-image (cwd) line,
container, stage dir, docker runtime, git remotes list, egress
allowlist summary, tls interception note, supervise note, prompt
metadata, remote-control flag.
`remote_control` kwarg kept on `.print()` for callsite stability
but unused in the compact format.
A `_multi(label, values)` helper does the "first value next to
the label, remainder continuation-indented" pattern that env /
skills / git gate / egress-proxy all share — keeps the columns
aligned to the label width.
Verified against my own dev bottle: output is byte-for-byte the
spec the operator asked for.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The apex-vs-subdomain question, the cert/SNI mismatch when
pipelock-passthrough hosts have wildcard certs, and the
mirror-divergence corner cases stacked up faster than the feature
earned its keep. Going back to exact-host match only.
Addon (`match_route`): single pass, case-insensitive exact match.
`*.foo.com` in a route table is now a literal string that won't
match anything — operators that want subdomains declare them
individually.
Pipelock mirror (`_pipelock_safe_hosts`): silently drops hosts
that don't fit pipelock's `[A-Za-z0-9_.-]+` charset (wildcards,
IPv6 literals, stray chars). Previously normalised wildcards to
their suffix; now just drops them, which matches egress-proxy's
behavior of not matching them either.
8 wildcard test cases removed; 2 lightweight "wildcards are not
supported" assertions retained as documentation. 386 unit pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`*.example.com` now matches `example.com` itself in addition to
every subdomain. RFC 6125 TLS-wildcard semantics excluded the
apex; an allowlist's natural reading of `*.example.com` is "all
of example.com" — and the pipelock mirror already strips
`*.example.com` to `example.com`, so without the apex match the
two layers disagreed (pipelock allowed the apex, egress-proxy
blocked it).
Behavior:
- `*.example.com` matches `example.com` (apex)
- `*.example.com` matches `foo.example.com` (subdomain)
- `*.example.com` matches `a.b.example.com` (nested)
- `*.example.com` does NOT match `barexample.com` (label
boundary required)
Test renamed: `test_wildcard_does_not_match_apex` →
`test_wildcard_matches_apex`. 395 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PRD 0017 v1 deliberately punted wildcards ("Exact match in v1 —
globs / wildcards are a follow-up"). Now that the supervise mirror
strips `*.` to its suffix for pipelock, the addon needs to actually
match wildcard hosts on its side or the route is dead weight.
Addon `match_route` now does two passes:
1. Exact (case-insensitive) literal match on the hostname.
2. Wildcard suffix match: a route whose host starts with `*.`
matches any request host that ends with `.<suffix>`. So
`*.example.com` matches `foo.example.com` and
`a.b.example.com`, but NOT the apex `example.com` and not
`barexample.com` (the leading `.` of the suffix is
required).
Exact wins — operators can layer a specific route (e.g.
`api.github.com` with auth) on top of a broader wildcard (e.g.
`*.github.com` bare-pass).
8 new unit tests: direct subdomain match, nested subdomain match,
apex rejection, overlapping-suffix rejection, case-insensitive,
exact-wins-over-wildcard (both route orders), no-match
fall-through. 395 unit + integration pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previous fix stripped wildcard hosts entirely from the pipelock
mirror; the operator wanted the suffix kept so pipelock pins the
base hostname. Now `*.example.com` becomes `example.com` in the
mirror — egress-proxy keeps the wildcard for its own host match,
pipelock allows the suffix.
Behavior change:
- `*.example.com` → `example.com` (was: dropped)
- `*.foo.bar.com` → `foo.bar.com` (one `*.` strip, not
recursive)
- `*` → dropped (normalises to empty)
- `example.com` → `example.com` (unchanged)
- `[::1]`, etc. → dropped (still off pipelock's
charset after any prefix
strip)
Adds explicit de-dup so `*.example.com` + `example.com` collapse
to one entry. Existing wildcard-strip test reshaped + 3 new
edge-case tests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pipelock's allowlist parser only accepts `[A-Za-z0-9_.-]+`
literal hostnames. Wildcard routes (`*.example.com`) that
egress-proxy's route table accepts trip pipelock's parser the
moment the mirror tries to render them into the new yaml; the
whole apply fails before pipelock is even touched. Symptom:
operator approves an egress-proxy-block proposal, gets
"pipelock allowlist mirror failed: allowlist line N: '<wildcard>'
has disallowed characters."
Fix: `_mirror_hosts_to_pipelock` filters through
`_pipelock_safe_hosts` before merging — anything outside
pipelock's allowed charset is silently skipped. Wildcard routes
stay live on egress-proxy; pipelock just won't pin a hostname
for the wildcard-matched traffic (caller's call to accept the
hostname-only enforcement gap there).
Adds 4 unit tests covering normal hostnames pass-through,
wildcard stripping, IPv6-literal stripping, and order
preservation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_mirror_hosts_to_pipelock` runs BEFORE the egress-proxy write in
`apply_routes_change` — if it raises, egress-proxy is left intact.
The error message claimed the opposite ("egress-proxy routes
updated but pipelock allowlist mirror failed"), pointing the
operator at the wrong half-state.
Reword to make the actual state clear: pipelock failed,
egress-proxy NOT updated, fix pipelock manually with
`pipelock edit <bottle>` then retry.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Instead of asking the agent to compose and submit a full routes
file, the tool now takes ONE proposed route — host + optional
path_allowlist + optional auth — and the supervisor merges it
into the live routes table at approval time. The agent no longer
needs to fetch / reproduce / extend the existing allowlist; it
just describes the host it wants reachable.
Tool input (new):
- `host` (required)
- `path_allowlist` (optional, array of absolute path prefixes)
- `auth` (optional, {scheme, token_ref})
- `justification` (required)
Merge semantics (in `egress_proxy_apply._merge_single_route`):
- Host NOT in current routes → append the proposed route as a
new entry. If `auth` is set, assign the next EGRESS_PROXY_TOKEN_N
slot.
- Host already present → union the proposed `path_allowlist`
with the existing one (proposed entries appended after
existing, deduped). Existing `auth_scheme` / `token_env`
preserved; proposed `auth` ignored (operator-controlled, not
agent-controlled).
- Hostname comparison is case-insensitive.
Dashboard wiring: `approve()` on an egress-proxy-block proposal
now calls `add_route(slug, proposed_route_json)` instead of
`apply_routes_change(slug, full_file)`. add_route fetches the
current routes from the running egress-proxy, merges, and calls
apply_routes_change with the merged content — so the
pipelock-mirror + SIGHUP plumbing from chunk 3 still runs
end-to-end. Audit diff still captures the full-file before/after.
Tool description rewritten to make the new shape obvious and to
stop pointing the agent at the routes file. The
`list-egress-proxy-routes` tool stays available for agents that
want to see what's currently allowed.
Tests: 9 new `_merge_single_route` cases (host absent/present,
path-allowlist union+dedup, auth-slot indexing, case-insensitive
match, existing-auth preservation, missing-host rejection,
malformed-current rejection). 407 unit + integration pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reshape the allowlist topology so the egress-proxy is the bottle's
single allowlist surface, and replace the agent-side
routes/allowlist file mounts with a live MCP tool.
Policy change (move defaults to egress-proxy):
- `egress_proxy_routes_for_bottle(bottle)` now folds in
DEFAULT_ALLOWLIST (the claude-code defaults) and
`bottle.egress.allowlist` (user adds) as bare-pass routes (no
auth, no path filter), on top of the bottle's
`egress_proxy.routes`. Manifest routes win on host collision.
- `pipelock_effective_allowlist(bottle)` mirrors egress-proxy's
effective host set when egress-proxy is in use. Pipelock is
no longer the bottle's primary allowlist authority; it
enforces a downstream copy as defense-in-depth + does DLP body
scanning.
- Split out `egress_proxy_manifest_routes(bottle)` for callers
that want just the manifest entries (tests, internal use).
- DEFAULT_ALLOWLIST moves from `pipelock.py` to `egress_proxy.py`
(pipelock re-imports for the no-egress-proxy fallback path).
- Dropped the `egress-proxy` auto-allow on pipelock's allowlist
— the agent never dials egress-proxy via the proxy mechanism;
pipelock only sees upstream hostnames from egress-proxy's
CONNECTs.
Introspection endpoint (existing mitmproxy feature):
- Egress-proxy addon recognises requests to the magic host
`_egress-proxy.local` and synthesizes responses via
`flow.response = http.Response.make(...)` — no upstream
connection, no allowlist enforcement on the magic host.
- `GET /allowlist` returns the in-memory route table as JSON
(host + path_allowlist + auth_scheme + token_env per route;
no token VALUES).
- Smoke-tested end-to-end against a real egress-proxy container.
MCP tool (existing supervise plumbing):
- New `list-egress-proxy-routes` tool (no inputs, no operator
approval). Handler fetches via egress-proxy's introspection
endpoint using urllib's ProxyHandler against
`EGRESS_PROXY_FORWARD_PROXY`. Returns the JSON payload as the
tool's text content; `isError: true` if the proxy is
unreachable.
- `egress-proxy-block` description now points the agent at
`list-egress-proxy-routes` instead of a staged file path.
- `pipelock-block` description acknowledges the mirror — agents
should prefer `egress-proxy-block` to add hosts; pipelock-block
stays for the rare divergence case.
Drop agent-side file mounts:
- Supervise's `current-config` dir staging no longer writes
routes.yaml / allowlist. Only `Dockerfile` remains
(capability-block still reads it from
`/etc/claude-bottle/current-config/Dockerfile`).
- `prepare.py` stops passing `routes_content` /
`allowlist_content` to `supervise.prepare`.
- `Supervise.prepare` signature simplified to one
`dockerfile_content` kwarg.
Tests: 400 unit + integration pass. Added coverage for
defaults-folding (`TestRoutesForBottleFoldsDefaults`), the new
tool definition + handler, and the updated supervise.prepare
shape.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When the operator approves an egress-proxy-block proposal that
adds a host to egress-proxy's routes, the request would still 403
downstream at pipelock — pipelock's hostname allowlist is set at
bottle launch and doesn't learn about routes added later. The
agent saw "Approved" but the very next retry still failed.
Fix: `apply_routes_change` now mirrors every host in the proposed
routes onto pipelock's allowlist before flipping egress-proxy.
Order matters — pipelock first so a pipelock failure doesn't
leave egress-proxy in a half-state:
1. Validate the new routes content.
2. Extract the hosts.
3. Merge them onto pipelock's current allowlist
(`apply_allowlist_change` — restarts pipelock with the merged
yaml). No-op when every host is already present.
4. docker cp the new routes.yaml into egress-proxy + SIGHUP.
If pipelock's restart fails, egress-proxy is untouched and the
operator gets a clear error pointing at the pipelock half-state.
If egress-proxy's update fails after pipelock succeeded, pipelock
just has the host pre-allowlisted — harmless extra-permissive
until the operator retries.
Adds `_hosts_in_routes` helper using the addon's own parser
(so the mirrored host set matches exactly what the addon will
match on). 4 new unit tests; 368 total pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
apply_routes_change wrote the proposed routes via
`tempfile.mkstemp` (default mode 0600) then `docker cp`'d into the
egress-proxy container. docker cp preserves mode + host uid, so
the file landed inside the container as 0600 owned by the host
user's uid — which is not the mitmproxy user (uid 1000) the
addon runs as. The SIGHUP-triggered reload then failed with
PermissionError on the re-read, the old routes table stayed in
memory, and the operator-approved route never took effect.
Symptoms reported:
- Operator approves egress-proxy-block proposal that adds
google.com to routes.
- Agent retries `curl https://google.com` and still gets 403
"egress-proxy: host 'google.com' is not in the bottle's
egress_proxy.routes allowlist."
- `docker exec <egress-proxy> cat /etc/egress-proxy/routes.yaml`
returns "Permission denied" (mitmproxy user can't read it,
so the reload couldn't either).
Fix: chmod 0644 on the host tmp file before docker cp. Mirrors
the same pattern in DockerEgressProxy.start which already chmods
the original routes.yaml + the CAs before cp. The proposed routes
content carries no secrets (tokens live in the egress-proxy
container's environ, not the routes file), so 0644 in /tmp for
the brief window between write and cp is safe.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The supervise sidecar mounted a snapshot named routes.json into
the agent at /etc/claude-bottle/current-config/routes.json, but
the egress-proxy-block tool description (and the live proxy file
the apply step writes) say routes.yaml. The agent couldn't find
the file at the documented path, composed proposals against stale
or empty current state, and reported "routes wasn't updated on
disk" because it was looking at the wrong filename.
Rename the staged file to routes.yaml so the tool description,
the staged snapshot, and the live proxy file all agree on the
name. Content stays JSON-in-a-yaml-extension (per PRD 0017
chunk 1's decision: every JSON document is valid YAML, stdlib
parsers handle it on both ends).
Note: the staged file is still a one-shot snapshot taken at
bottle prep time. It does NOT auto-update when the operator
approves an egress-proxy-block. Agents that want to verify
their proposal took effect should retry the request that
triggered the block — a successful upstream response is the
real signal. Fixing the snapshot-staleness UX is a separate
follow-up.
Tests migrated from routes.json → routes.yaml. 364 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CVE-2016-5388 ("httpoxy") mitigation: libcurl ignores uppercase
HTTP_PROXY for http:// URLs to prevent untrusted CGI HTTP_*
headers from hijacking the proxy. Only lowercase http_proxy is
honored for HTTP. Without the lowercase var, plain-HTTP requests
from the agent skip egress-proxy entirely — they go direct,
which is "network unreachable" on the agent's --internal bridge,
not the egress-proxy 403 we expect.
Confirmed against a live bottle: `curl http://1.1.1.1/` reported
"Immediate connect fail for 1.1.1.1: Network is unreachable"
instead of the addon's "host not in allowlist" 403. With both
cases set the agent's curl honors the proxy and our allowlist
enforcement kicks in.
Also set lowercase HTTPS_PROXY + NO_PROXY for symmetry. Some
tools check one case only; sending both means we don't have to
audit which convention each tool uses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two issues stopping the bottle's egress allowlist from being
enforced:
1. mitmproxy was bypassing pipelock. We set HTTPS_PROXY=pipelock
in the egress-proxy container's env, but mitmproxy is a proxy
*server* — it does NOT honor HTTP(S)_PROXY env vars on its
outbound side the way HTTP-client libraries do. All
post-MITM traffic was going direct to the upstream, never
touching pipelock's hostname allowlist or DLP scanner.
Fix: use mitmproxy's `--mode upstream:URL` flag. The Dockerfile
entrypoint now reads a new `EGRESS_PROXY_UPSTREAM_PROXY` env
(set by `DockerEgressProxy.start` to the pipelock URL when
pipelock is in the topology) and switches mitmdump to
upstream-proxy mode. Standalone runs of the image without the
env still get `--mode regular@9099` direct-to-upstream — useful
for unit-test boots. Confirmed in the boot log: "HTTP(S) proxy
(upstream mode) listening at *:9099."
2. egress-proxy was forwarding unrecognized hosts. The addon's
`decide()` returned `Decision(action="forward")` whenever no
route matched the request host, deferring to pipelock to gate.
With #1 broken pipelock wasn't gating either; even with #1
fixed, defense-in-depth wants both layers enforcing.
Fix: no-route-match → 403 with a "host not in allowlist"
reason. The egress allowlist is now strictly the set of hosts
declared in `bottle.egress_proxy.routes`; bare-pass routes
(host with no auth, no path_allowlist) cover the passthrough
case for hosts that just need reach. path_allowlist enforcement
on matched routes is unchanged.
Test updated: `test_no_matching_route_forwards` →
`test_no_matching_route_blocks`. 364 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of the persistent SSL handshake failure: pipelock's
`tls init` stamps a non-standard `Subject Key Identifier` on the
CAs it generates (random rather than SHA-1 of the pubkey).
mitmproxy computes the `Authority Key Identifier` on each leaf
cert it mints as SHA-1(issuer's pubkey). openssl's chain validator
uses the leaf's AKI to find the issuer cert by SKI; with pipelock's
SKI off by definition, the lookup fails and openssl returns
"unable to get local issuer certificate" — even though the CA is
right there in the trust store with the matching SHA-256
fingerprint. (Also, pipelock generates EC CAs; the cert+key concat
fit in 834 bytes vs ~2.3KB for RSA, which was the first red
flag.)
Diagnostic from a live bottle confirmed:
leaf cert AKI: A8:F0:D5:E3:B5:B9:C2:38:2B:9F:DD:4A:DF:26:8C:72:19:A2:5E:94
CA cert SKI: 81:CA:6D:4C:ED:5C:C2:B1:48:0C:3E:E8:8D:73:86:97:B9:89:B4:3D
CA cert + leaf cert: same Pipelock-named subject, same public key bytes
openssl verify -CAfile <our CA> <leaf>: error 20
Fix: switch `egress_proxy_tls_init` from `pipelock tls init` to
host `openssl req` with an explicit `subjectKeyIdentifier=hash`
extension. SHA-1(pubkey) for the SKI matches what mitmproxy puts
in the AKI, so chain validation works. The generated CA is also
RSA-2048 / sha256WithRSAEncryption — mitmproxy's most-tested
configuration.
The new generator is independent of pipelock entirely (no docker
run on the pipelock image to mint the CA), so the egress-proxy
CA generation now requires only `openssl` on the host. macOS +
Linux dev images both have it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`--set ssl_verify_upstream_trusted_ca` REPLACES mitmproxy's default
trust store with the file we point it at. The earlier wiring
pointed it at just pipelock's CA, which broke for any host pipelock
passes through (api.anthropic.com is in DEFAULT_TLS_PASSTHROUGH):
pipelock CONNECT-tunnels the handshake to the real upstream,
egress-proxy sees the real public cert (signed by e.g. DigiCert),
and refuses to validate because pipelock's CA doesn't sign it.
Fix in Dockerfile entrypoint: when EGRESS_PROXY_UPSTREAM_CA is
set, concatenate /etc/ssl/certs/ca-certificates.crt + the pipelock
CA into /home/mitmproxy/.mitmproxy/combined-trust.pem, and pass
that as ssl_verify_upstream_trusted_ca. Covers both legs:
- pipelock-MITM'd hosts → leaf cert signed by pipelock CA →
validates against the pipelock half of the bundle.
- pipelock-passthrough hosts (api.anthropic.com et al.) → real
upstream cert → validates against the system half.
Standalone runs of the image (no EGRESS_PROXY_UPSTREAM_CA) skip
the concat and use mitmproxy's default trust store.
Reproduces against today's main: agent gets "Unable to connect to
API: SSL certificate verification failed" on api.anthropic.com,
egress-proxy logs "Server TLS handshake failed. Certificate verify
failed: unable to get local issuer certificate". After this patch
the trust bundle includes the real upstream root + pipelock's CA
and both validation paths succeed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis