Phase 4 of PRD 0016. End-to-end test against real Docker:
- Stages a fake bottle: alpine:latest container named
claude-bottle-<slug> with a marker file at
/home/node/.claude/sessions.json, plus a fake supervise sidecar.
- Calls apply_capability_change with a new Dockerfile.
- Verifies: per-bottle Dockerfile written, agent + sidecars
removed, networks removed, transcript snapshot dir on host
contains the marker file (proving docker cp transferred bytes).
- Subsequent-apply test proves the per-bottle Dockerfile state
persists across rebuilds (before-diff uses the prior override,
not the repo Dockerfile).
- Teardown-idempotent test: apply against a never-started bottle
doesn't raise.
docker exec / cp / rm / network rm work fine across the docker
socket boundary, so this runs in DinD too — no act_runner skip
needed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 4 of PRD 0015. End-to-end test against real Docker:
- Brings up a real pipelock sidecar via the production
DockerPipelockProxy bring-up + pipelock_tls_init.
- Calls apply_allowlist_change to add a new host.
- Polls the live /etc/pipelock.yaml until the new host shows up
(bridging the docker-restart window).
- Verifies api_allowlist contains both old + new hosts and
tls_interception block is preserved.
- Smaller cases: invalid hostname raises, missing sidecar raises,
fetch_current_allowlist returns one-per-line format.
Skipped under GITEA_ACTIONS because pipelock_tls_init bind-mounts a
host path that doesn't share fs in the runner, matching the
existing pipelock smoke test's skip pattern.
Drive-by fix: fetch_current_yaml now uses `docker cp` (daemon-API
tarball copy) instead of `docker exec cat` because the pipelock
image is distroless and has no shell utilities.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 5 of PRD 0014. End-to-end test against real Docker:
- Brings up a cred-proxy sidecar with route /a/ → unreachable
upstream (so 502 = route matched, 404 = no route).
- Calls apply_routes_change to swap to /b/ only.
- Polls until the route table flips: /a/ now 404s, /b/ now 502s.
- Separately verifies fetch_current_routes returns the live file,
apply with invalid JSON raises, and apply against a non-existent
sidecar raises.
No fake-upstream container needed: unreachable hostnames give the
502 signal directly. apply_routes_change uses docker exec / cp / kill
(not bind mounts), so this should work in docker-in-docker too —
no DinD skip needed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The integration test test_tools_call_round_trips_through_queue
relies on a host bind-mount to share the queue dir between the
sidecar (writing proposals) and the test process (approving via
dashboard helpers). In the Gitea Actions runner the docker socket
forwards to the outer host's daemon, so bind-mount paths are
resolved against the outer host's fs — not the runner container's.
The sidecar writes its proposal where the test can't see it; the
test times out.
Add a one-shot probe that does docker run -v <tmp>:<container> and
checks both directions of fs visibility. Skip the round-trip test
when the probe fails. tools_list and the orphan-name test are
unaffected — they don't touch the queue.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 5 of PRD 0013. End-to-end integration test against real Docker:
- Brings up the supervise sidecar on a per-bottle internal network.
- A curl-image "agent" on the same network does tools/list and gets
back the three PRD 0013 tool names over real MCP wire format.
- A tools/call round-trips through the queue: agent blocks on the
call, host watches the queue, dashboard.approve writes a Response,
agent receives the approval payload (status, notes) in MCP content.
- Documents the orphan-sidecar name-collision behavior so a future
auto-cleanup change can flip the assertion.
Skips if docker is unreachable, matching the existing integration
pattern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Manifest.resolve walks $HOME/.claude-bottle/{bottles,agents}/ and
$CWD/.claude-bottle/agents/ instead of reading claude-bottle.json.
A bottles/ subdir under $CWD is logged as a warn and ignored —
the filesystem layout IS the trust boundary, no resolver check
needed.
If claude-bottle.json exists alongside no .claude-bottle/ dir at
either location, dies with a clear pointer at the README — the
manifest format changed and we don't silently fall back.
Manifest.from_md_dirs(home, cwd) is the programmatic entry point
tests use to build a Manifest from fixture directories without
touching os.environ. Manifest.from_json_obj is preserved for
tests that still want to build manifests in-memory.
Bottle / agent frontmatter goes through Bottle.from_dict /
Agent.from_dict — same validators as today's JSON path. Unknown
top-level frontmatter keys die with a "did you mean" pointer
listing accepted keys. Filenames that don't match [a-z][a-z0-9-]*
are skipped with a warn.
Agent files accept the Claude Code subagent passthrough fields
(name, description, model, color, memory) so the same file can
drop into ~/.claude/agents/ — claude-bottle ignores them at
launch but doesn't reject.
The dry-run integration test ships a real MD fixture tree now;
all 200 unit + 17 integration tests stay green.
Removes the legacy `CLAUDE_BOTTLE_OAUTH_TOKEN` -> `CLAUDE_CODE_OAUTH_TOKEN`
forward in prepare.py. Bottles that need claude-code to authenticate
must declare a cred_proxy route with role: "anthropic-base-url" — there
is no fallback that hands the token to the agent directly.
Drops the now-dead BottleSpec.forward_oauth_token field, the CLI
setter that read CLAUDE_BOTTLE_OAUTH_TOKEN from the host env at
prepare time, and the forward_oauth_token=False arg in the six
pipelock integration tests.
PRD 0010 and README updated; the dev ~/claude-bottle.json gains an
anthropic-base-url route so the implementer/researcher agents keep
working.
BREAKING: bottles previously relying on the implicit OAuth forward
will now produce an agent environ without any Anthropic credential.
Verified with --dry-run: a bottle with no anthropic-base-url route
yields env_names: [] (no token at all); a bottle that declares the
route yields ANTHROPIC_BASE_URL plus a non-secret placeholder for
CLAUDE_CODE_OAUTH_TOKEN.
Three leftovers from the manifest refactor:
1. provision/cred_proxy.py:223 referenced u.kind == 'gitea' for the
tea login count — kind was removed from the runtime class, so any
bottle with a tea-login route raised AttributeError at provision
time. Switch to `'tea-login' in r.roles`.
2. The runtime class CredProxyUpstream is renamed to CredProxyRoute
(its data is a route on the proxy, not an "upstream"; the field
route.upstream is the upstream URL). Module's own naming now
aligns with manifest.CredProxyRoute and routes.json.
3. cred_proxy_upstreams_for_bottle -> cred_proxy_routes_for_bottle;
CredProxyPlan.upstreams -> CredProxyPlan.routes; local
`upstreams` collections become `routes`. Callers in
backend.py, launch.py, prepare.py, bottle_plan.py,
provision/cred_proxy.py, and tests updated.
Also strips lingering `bottle.tokens` references from docstrings
(pipelock.py, cred_proxy.py prepare(), manifest._parse_https_host,
test_pipelock_allowlist.py module doc) and removes dead helpers
from the integration test (the _bottle helper used a tokens field
that no longer parses).
Replace bottle.tokens (with Kind enum and hardcoded per-kind
route/auth tables) with bottle.cred_proxy.routes — each route
declares its own path, upstream, auth_scheme, token_ref, and
optional role[]. The manifest is now the source of truth for the
proxy's runtime route table; adding an upstream is a manifest edit,
not a code change.
Agent-side rewrites move from per-kind dispatch to per-role tags
on routes:
anthropic-base-url -> set ANTHROPIC_BASE_URL=<proxy><path>
npm-registry -> write ~/.npmrc registry=
git-insteadof -> write ~/.gitconfig [url] insteadOf, keyed
off route.upstream (suppressed when
bottle.git brokers the same host)
tea-login -> add a ~/.config/tea/config.yml login
Roles are a list (string accepted as sugar). A gitea route
typically carries ["git-insteadof", "tea-login"]. Singleton roles
(anthropic-base-url, npm-registry) appear on at most one route.
token_env slots are assigned per distinct TokenRef in declaration
order — two routes sharing a token_ref (e.g. github API + git
endpoints) share a slot.
Drops: TOKEN_KINDS, _KIND_ROUTES, _KIND_AUTH_SCHEME, _TOKEN_DEFAULT_HOST,
cred_proxy_route_path_for_gitea, the kind field on CredProxyUpstream,
and the kind-based hardcoding in pipelock_token_hosts (now derives
from route.UpstreamHost).
Legacy bottle.tokens manifests now die with a hint pointing at
bottle.cred_proxy.routes + this PRD. Tests rewritten end-to-end.
Docs + example.json + the dev ~/claude-bottle.json updated to match.
Drives DockerCredProxy.start through the production code path against
a fake upstream container running on the same egress network. The
"agent" is a curl container on the bottle's internal network — same
access topology the agent uses in production.
Covers PRD 0010 success criteria:
- SC3 (the request reaches upstream, header round-trip works)
- SC6 (inbound Authorization stripped; the proxy injects the
configured token even when the agent tries to smuggle one in)
- partial SC2 (cred-proxy reachable by the alias from the internal
network)
- 404 for unconfigured routes
Live-network tests against real Anthropic / GitHub / Gitea / npm
upstreams (SC4 and SC5 specifically) are deferred — the fake-upstream
shape covers the routing + header layer that's actually under test
here.
- Delete tests/unit/test_ssh_gate.py and the fixture_with_ssh helpers.
- test_pipelock_yaml: drop the ssh-leak guard (structurally
impossible now); the remaining tests switch to fixture_minimal.
- test_pipelock_allowlist: rewrite the union/dedup test to
exercise an egress.allowlist that duplicates a baked default
(the property the ssh-leak assertion was hitching onto).
- test_manifest_git: shadow-route assertion becomes a legacy-ssh-
dies-with-hint assertion, since bottle.ssh is now parse-fail.
- test_orphan_cleanup: drop the SSHGate.stop idempotency check;
pipelock equivalent stays.
- test_dry_run_plan: drop assertions on the removed ssh_hosts /
ssh_gate keys.
52 unit tests pass.
A pair of integration tests against a real sshd-based "upstream"
sibling container that prove every operation through the gate is
observably equivalent to the same operation against the upstream:
- test_clone_and_refetch_reflect_upstream: clone via gate
returns the upstream's current commit; an out-of-band commit
on the upstream shows up via the gate on the next ls-remote.
- test_push_through_gate_lands_on_upstream: a clean push routed
through the gate lands on the upstream's bare repo.
The upstream container is a tiny inline-built alpine image with
openssh-server, a `git` user (passwd -u so sshd doesn't reject
the locked account), and a baked bare repo seeded with one
commit. Host keys are baked in at build so the test can pin
KnownHostKey on the manifest entry before the container starts.
While wiring this up the access-hook gained a one-shot HEAD
sync: `git init --bare` defaults HEAD to refs/heads/master, and
upstreams that use main would leave the bare repo's HEAD
unresolvable — clones came through but the working tree was
empty. The hook now does a `rev-parse --verify HEAD` check
after the first fetch and runs `ls-remote --symref` to repoint
HEAD if it doesn't resolve. One extra round-trip on first
fetch only.
The gate is now a transparent mirror, not push-only. Per-repo
init now runs `git remote add --mirror=fetch origin <url>` so a
later `git fetch origin` mirrors the upstream's full ref graph at
canonical paths. The pre-receive hook forwards accepted refs via
`git push origin` (renamed from upstream).
New: an access-hook script wired via `git daemon --access-hook`
runs `git fetch origin --prune` against the real upstream before
every upload-pack request (clone, fetch, pull, ls-remote). On
upstream error the hook exits non-zero — the agent's fetch fails
rather than the gate serving stale data.
The pre-existing smoke test (ls-remote against unreachable
upstream returns refs) had to invert: under the bidirectional
design any ls-remote success is necessarily a success against
the upstream, so the unreachable-upstream case now correctly
fails closed.
Two integration tests against a real Docker daemon:
- test_ls_remote_succeeds_against_fresh_gate: a freshly-started
gate has its empty bare repo exported via git daemon; ls-remote
from a sibling container on the internal network returns no
refs and exits 0.
- test_push_with_secret_is_rejected: the PRD 0008 success
criterion — a push containing an AKIA-shaped synthetic that
trips gitleaks's aws-access-token rule is rejected by the
pre-receive hook with a non-zero exit on the client and a
gitleaks rejection in the response.
Dockerfile.git-gate switches base to zricethezav/gitleaks (alpine
3.22 + gitleaks v8.30.1, pinned by digest) since gitleaks isn't
packaged for alpine, and adds git-daemon (the sub-package the
listener needs; the core git binary in the base doesn't include
the daemon).
DockerBottleBackend now instantiates a DockerGitGate alongside
DockerPipelockProxy and DockerSSHGate; the prepare step lifts
bottle.git into a GitGatePlan stored on DockerBottlePlan, and
launch starts/stops the sidecar in the same ExitStack as the
other two (only when bottle.git is non-empty).
bottle_plan.print now surfaces git remotes and per-upstream gate
forwards in the y/N preflight; to_dict adds git_remotes and
git_gate keys to the dry-run JSON payload for CLI consumers.
PRD: docs/prds/0008-git-gate.md
Pipelock's BIP-39 seed-phrase scanner fires on Anthropic Messages API
bodies because user-authored conversation text can hit 12 consecutive
BIP-39 dictionary words that pass the checksum, returning a 403
`blocked: request body contains secret: BIP-39 Seed Phrase` that the
Claude CLI surfaces as `Please run /login`. Pipelock's `suppress`
section only covers git/file findings, not the inline body scanner,
so the recommended treatment for LLM endpoints is
`tls_interception.passthrough_domains`: CONNECT is still allowlist-
gated, but the body is not MITM'd. The existing body-scan integration
test moves to `raw.githubusercontent.com` so it still pins TLS body
DLP on non-passthrough'd hosts.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PRD 0007: the launch ExitStack calls gate.stop on every failure
path, so an early bring-up error (where the gate container was
never created) must not raise from teardown. Mirrors the existing
DockerPipelockProxy.stop assertion.
The orphan-container enumeration in cleanup.py already covers
ssh-gate containers via its `claude-bottle-` name prefix filter —
no code change there.
PRD 0007: thread the DockerSSHGate through the bottle lifecycle.
- DockerBottlePlan gains gate_plan: SSHGatePlan.
- prepare.resolve_plan accepts a gate and renders its entrypoint
script next to the pipelock yaml.
- launch.launch starts the gate sidecar after pipelock (so it's on
the same internal + egress networks) and registers its stop in
the ExitStack. Skipped when the bottle has no ssh entries.
- DockerBottleBackend instantiates DockerSSHGate alongside the
pipelock proxy.
- bottle_plan.print + to_dict surface the upstream table so
--dry-run shows the per-host listen-port mapping.
ssh_config provisioning still points at pipelock; that swap lands
in the next commit so this one stays a pure wiring change.
Fourth and final step of PRD 0006. Two new end-to-end tests pin
the two paths through pipelock's tls_interception layer.
- test_pipelock_blocks_secret_https_post: posts a GitHub-PAT-shaped
body to api.anthropic.com over HTTPS through the bottle. With
pipelock now bumping the CONNECT and seeing the decrypted body,
it returns 403 with the documented `blocked: request body
contains secret: GitHub Token` body. The probe is a single curl
invocation — curl natively does CONNECT through HTTPS_PROXY, the
agent's trust store now contains pipelock's CA, no hand-rolled
TLS in the test.
- test_pipelock_allows_normal_https: GETs git's README from
raw.githubusercontent.com (a baked-in allowlist host). 200 +
non-zero body length proves the full chain works:
pipelock_tls_init → docker cp of CA into sidecar → bumped CONNECT
→ provision_ca installed CA in agent → curl trusts pipelock's
bumped leaf → body forwarded back through the tunnel.
- test_pipelock_sidecar_smoke: pre-existing direct-start smoke
test updated to call pipelock_tls_init and populate the CA
paths on the plan. (The full launch flow does this in launch.py;
this test exercises the proxy class in isolation.)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Third step of PRD 0006. The preflight now surfaces the TLS-
intercept layer so the operator sees it before agreeing to launch.
- Text output: one new line under the egress summary
("tls intercept : pipelock (per-bottle ephemeral CA, generated
at launch)").
- JSON output (--format=json contract): new
egress.tls_interception: { enabled: true, ca_fingerprint: null }
block. Fingerprint is always null at dry-run because the CA
only exists after launch; real launches print it as a stderr
log line from provision_ca.
- Pin the new shape in the dry-run integration test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The no-side-effects assertion calls `docker network ls` and
`docker ps -a` to verify the dry run created nothing. Inside the
Gitea Actions job container, those exit non-zero against the
host-mounted docker socket — the same act_runner topology issue
that already excludes other integration tests from CI (see
docs/ci.md). The failure was silently swallowed under the default
check=False; the recent style sweep that added check=True surfaced
it.
Gate the docker-enumerating check on GITEA_ACTIONS so the JSON
contract — the more useful part of the test — keeps running on CI.
Consolidate the two count helpers into one that surfaces stderr in
the failure message instead of raising a context-free
CalledProcessError, so the next docker surprise is debuggable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds bottle.egress.dlp_action ("block" | "warn", default block) and
wires it into pipelock as request_body_scanning.action. Pipelock's
own default is "warn", which previously meant claude-bottle detected
credential patterns in outbound bodies but forwarded the request
anyway.
The matching integration test posts a manifest env var shaped like
a GitHub PAT to api.anthropic.com via plain HTTP forward proxy so
pipelock can see the body. Pipelock answers 403 from its body-scan
layer instead of forwarding to the upstream.
Behavior change: bottles without an explicit egress.dlp_action now
block on body-scan hits. Set egress.dlp_action: "warn" to restore
the prior detect-only behavior.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bottle.exec(script) -> ExecResult runs a POSIX shell script inside a
running bottle and returns captured stdout/stderr/returncode. The
Docker impl pipes the script via stdin to `docker exec -i ... sh -s`
so the source never crosses argv.
Two integration tests exercise it end-to-end through the pipelock
sidecar: a Node request to a non-allowlisted host (example.com)
returns 403 from pipelock; a Node CONNECT to an allowlisted host
(raw.githubusercontent.com) is tunneled with 200 Connection
Established. The 200/403 split on each verb is decided by pipelock
itself, isolating the allowlist decision from whatever the remote
might return.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The test overrides HOME to isolate the manifest under test from the
dev's real ~/claude-bottle.json. On Docker Desktop that override
also breaks docker CLI endpoint resolution, since the active context
is read from $HOME/.docker/config.json and the per-user socket lives
under $HOME/.docker/run/docker.sock. Forward the parent's resolved
endpoint via DOCKER_HOST so the subprocess reaches the same daemon
regardless of $HOME.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Silences pylint W1510 / ruff PLW1510 across the codebase. The choice
at each site reflects existing intent:
- check=True where the caller implicitly trusts success (docker ps /
network ls returning stdout, docker build, exec chown/chmod inside
provisioners).
- check=False where the caller inspects .returncode (race-retry on
docker run, pipelock sidecar lifecycle, network plumbing, exec_claude
propagating the session's exit code, best-effort cleanup paths).
No behavior change; check= defaults to False so the False sites are
semantically identical.
The smoke test now drives the production prepare/start path, which
calls network_create_internal. Under Gitea act_runner the docker
socket mount topology makes `docker network create --internal` fail
(or be invisible across the host/job-container boundary) — the same
limitation that test_orphan_cleanup.test_create_and_remove already
skips for. Match that skip here so CI goes green; the test still
runs in environments with a direct docker daemon.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PipelockProxy.prepare now accepts (bottle, slug, stage_dir) and derives
the yaml_path itself, so callers don't need to know the filename.
DockerBottleBackend.prepare_proxy becomes a one-line wrapper whose only
caller already has bottle and slug in scope, so it's inlined and
deleted.
Move the --format=json-requires-dry-run check out of the integration
suite (it doesn't need Docker — argparse fails before any backend
runs) and tighten the assertion: previously asserted only that exit
code was nonzero, so any unrelated breakage (manifest resolution
failure, bad agent name, etc.) silently passed. Now asserts stderr
contains the actual flag-conflict message.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The old smoke test hand-rolled the docker create/cp/start sequence in
parallel with what DockerPipelockProxy.start already does, so any
divergence in production code wouldn't trip it. Rewritten to call
.prepare and .start directly and probe /health from a sibling curl
container on the same internal network — same access topology the
agent container uses in production.
In-network probing means the test no longer depends on a published
port, so it can run under act_runner (where host-loopback port
publishing isn't reachable from the job container).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
BottlePlan gains a to_dict method (abstract on the base, implemented
on DockerBottlePlan) returning a JSON-serializable view of the resolved
plan. `cli.py start --dry-run --format=json` prints it to stdout and
exits zero. --format=json without --dry-run is rejected — emitting JSON
during a real launch would race the y/N prompt.
The dry-run integration test now parses the JSON and asserts on
structured fields (agent, bottle, runtime, hosts sorted+deduped, etc.)
instead of regex-matching the human-readable preflight stdout. That
kills the magic-"8 hosts allowed" coupling — adding a new baked
default doesn't break the test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the hand-maintained INTEGRATION_NAMES classifier (and the
bespoke run_tests.py around it) with a directory-driven split:
tests/unit/ unit tests, always run
tests/integration/ Docker-dependent, skip cleanly without Docker
tests/canaries/ upstream-regression checks, opt-in via
CLAUDE_BOTTLE_RUN_CANARIES=1
The pinned-pipelock-image check moves to the canary suite — it tests
upstream packaging, not our code, so it shouldn't gate every dev push.
A scheduled canaries.yml workflow runs it weekly.
The manifest-runtime tests collapse the four assertRaises cases for
distinct 'runtime' values into one subTest loop and drop the
error-message-wording assertions; the contract is "any value is
rejected", not "the error literally contains 'auto-detect'".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces cli.sh + lib/*.sh with a claude_bottle/ Python package and a
cli.py entry point. No external dependencies — uses only Python's
stdlib (json, subprocess, getpass, tempfile, argparse, re, etc.).
- claude_bottle/{log,docker,manifest,env_resolve,network,pipelock,
skills,ssh,cli}.py mirror the previous lib/*.sh modules.
- Tests converted to unittest under tests/test_*.py with a stdlib
runner at tests/run_tests.py (unit | integration | path).
- .githooks/commit-msg ported to Python; same Conventional Commits rules.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis