didericis/bot-bottle
1
0
Fork 0
Code Issues 7 Pull Requests 15 Actions Packages Projects Releases Wiki Activity

PRD 0005: mitmproxy TLS interception for pipelock content scanning #8

Closed
didericis wants to merge 6 commits from mitmproxy-tls-interception into main
pull from: mitmproxy-tls-interception
merge into: didericis:main
Conversation 1 Commits 6 Files Changed 14
+1409 -14
didericis commented 2026-05-12 12:20:41 -04:00
Owner
Copy Link
Copy Source

Summary

Captures the design for putting a mitmproxy sidecar in front of pipelock on the egress path so pipelock's body / header / MCP scanners see plaintext for the HTTPS hosts in the default allowlist. Implements Topology A from docs/research/tls-mitm-for-pipelock.md (mitmproxy → pipelock → internet) with a per-bottle ephemeral CA, no manifest schema change in v1, and selective-bumping deferred until a pinning host appears.

Context

  • Pipelock today only sees CONNECT hostnames and opaque TLS bytes. The 48-pattern DLP layer is inert against every host in DEFAULT_ALLOWLIST (all HTTPS).
  • tests/integration/test_pipelock_blocks_secret_post.py only fires because it forces plain HTTP; real Claude Code traffic to api.anthropic.com slips past.
  • Research note: docs/research/tls-mitm-for-pipelock.md. Recommended Topology A: mitmproxy in front of pipelock.
## Summary Captures the design for putting a mitmproxy sidecar in front of pipelock on the egress path so pipelock's body / header / MCP scanners see plaintext for the HTTPS hosts in the default allowlist. Implements Topology A from `docs/research/tls-mitm-for-pipelock.md` (mitmproxy → pipelock → internet) with a per-bottle ephemeral CA, no manifest schema change in v1, and selective-bumping deferred until a pinning host appears. ## Context - Pipelock today only sees CONNECT hostnames and opaque TLS bytes. The 48-pattern DLP layer is inert against every host in `DEFAULT_ALLOWLIST` (all HTTPS). - `tests/integration/test_pipelock_blocks_secret_post.py` only fires because it forces plain HTTP; real Claude Code traffic to `api.anthropic.com` slips past. - Research note: `docs/research/tls-mitm-for-pipelock.md`. Recommended Topology A: mitmproxy in front of pipelock.
didericis added 1 commit 2026-05-12 12:20:41 -04:00
docs(prd): add 0005 mitmproxy TLS interception
test / unit (pull_request) Successful in 10s
Details
test / integration (pull_request) Successful in 13s
Details
fe71249005 
Captures the design for putting a mitmproxy sidecar in front of
pipelock on the egress path so pipelock's body / header / MCP
scanners see plaintext for the HTTPS hosts in the default allowlist.
Implements Topology A from docs/research/tls-mitm-for-pipelock.md
with a per-bottle ephemeral CA, no manifest schema change in v1,
and selective-bumping deferred until a pinning host appears.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis added 1 commit 2026-05-12 12:54:31 -04:00
docs(prd): update 0005 after open-question walkthrough
test / unit (pull_request) Successful in 17s
Details
test / integration (pull_request) Successful in 15s
Details
c2eacac49f 
Re-grounds the design after walking the eight original open
questions interactively. Two structural changes:

- Topology A → A'. A spike confirmed mitmproxy's `upstream` mode
  re-wraps decrypted flows in a new CONNECT to the upstream proxy,
  which would have left pipelock seeing only ciphertext (the very
  gap this PRD set out to close). The fix is to run mitmproxy in
  `regular` mode and ship a vendored Python addon that forwards
  each decrypted request to pipelock as a plain HTTP forward-proxy
  call. Pipelock is unchanged.
- mitmproxy owns CA generation. The research note's preference
  for a host-side openssl / cryptography CA turned out to be
  unnecessary — mitmproxy generates a fresh CA on startup; the
  public cert is `docker cp`'d into the agent. No new host-side
  crypto deps. Dry-run can't render a fingerprint (CA doesn't
  exist yet); launches print it once to stderr.

Other Q3–Q8 resolutions folded in: Debian-base `update-ca-certificates`
confirmed, mitmproxy 12 verified to speak h2 on both halves,
selective-bump deferred to v2, response-body and MCP scanning
deferred to v2, domain-fronting deferred to v2.

Open questions rewritten — what remains is addon-implementation
specifics (pipelock 403-body fingerprint, env-var inheritance
through docker exec, addon test fixtures).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis added 4 commits 2026-05-12 13:46:29 -04:00
feat(mitmproxy): vendor the addon and Docker sidecar lifecycle e579c3d4fd 
First step of PRD 0005. Three new files for the
mitmproxy-in-front-of-pipelock topology — wiring into the bottle
launch comes in the next commit.

- claude_bottle/mitmproxy/__init__.py: abstract MitmproxyProxy
  base + MitmproxyProxyPlan. Mirrors the PipelockProxy shape
  (prepare / start / stop) and adds extract_ca_cert for the CA
  cert hand-off into the agent.
- claude_bottle/mitmproxy/addon.py: the vendored Python addon
  mitmproxy loads inside the sidecar. Forwards each decrypted
  request to pipelock as a plain HTTP forward-proxy call,
  inspects the response, and short-circuits the flow with 403 on
  a pipelock block (status=403 + body starts with `blocked: `,
  pinned empirically against pipelock 2.3.0 in the impl spike).
  Self-contained — no claude_bottle imports — so it loads in a
  sidecar that doesn't have claude_bottle on its path.
- claude_bottle/backend/docker/mitmproxy.py: DockerMitmproxyProxy
  with create / cp / network connect / start lifecycle. Pinned
  to mitmproxy/mitmproxy@sha256:00b77b5d… (multi-arch manifest
  for v12.2.3).
- tests/unit/test_mitmproxy_verdict.py: pins the verdict
  fingerprint so a pipelock-side body shape change breaks loudly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feat(mitmproxy): wire the sidecar into the bottle launch lifecycle 21054212d4 
Second step of PRD 0005. The mitmproxy sidecar from the previous
commit now actually runs alongside pipelock when a bottle launches.

- BottleBackend gains a non-abstract provision_ca with a default
  no-op so non-Docker backends aren't forced to implement TLS
  interception. provision() orchestrates ca → prompt → skills → ssh
  → git; CA goes first so trust is set up before anything else runs
  inside the agent.

- DockerBottlePlan gains `mitmproxy_plan: MitmproxyProxyPlan`. The
  prepare step builds it alongside the existing pipelock plan; no
  new manifest schema or host-side scratch files.

- DockerBottleBackend grows self._mitm, threads it through prepare
  and launch. Mirror of the existing self._proxy pattern.

- launch.py brings the mitmproxy sidecar up between pipelock and
  the agent container, passing pipelock's service-name URL via
  env. ExitStack callback handles teardown in reverse order.

- The agent's HTTPS_PROXY / HTTP_PROXY now point at mitmproxy (not
  pipelock directly). Three new -e flags inject the CA trust trio
  (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE) at
  docker run time; Docker propagates those into docker exec so the
  claude process sees them without per-exec threading.

- New provisioner backend/docker/provision/ca.py extracts the CA
  cert from the running mitmproxy sidecar, copies it into the agent
  at /usr/local/share/ca-certificates/claude-bottle-mitm.crt, runs
  update-ca-certificates, and emits a stderr line with the SHA-256
  fingerprint (stdlib ssl + hashlib; no subprocess).

Cleanup needs no change — `docker ps --filter name=^claude-bottle-`
already catches the new claude-bottle-mitm-<slug> containers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feat(mitmproxy): render mitmproxy in the dry-run preflight c4de42ea3c 
Third step of PRD 0005. The preflight now surfaces the TLS-
intercept layer so the operator sees it before agreeing to launch.

- Text output: one new line under the egress summary —
  "tls intercept : mitmproxy (per-bottle ephemeral CA, generated
  at launch)".
- JSON output (--format=json contract): new
  egress.mitm: { enabled: true, ca_fingerprint: null } block.
  Fingerprint is always null at dry-run because the CA only
  exists after the sidecar starts; real launches print it as a
  stderr log line from provision_ca.
- Pin the new shape in the dry-run integration test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feat(mitmproxy): integration tests for the bumped HTTPS path
test / unit (pull_request) Successful in 20s
Details
test / integration (pull_request) Successful in 15s
Details
22bc13dc3c 
Fourth and final step of PRD 0005. Two new end-to-end tests that
exercise the full chain agent -> mitmproxy(bump) -> addon ->
pipelock -> upstream and pin the two paths the addon implements.

- test_mitmproxy_blocks_secret_https_post: HTTPS variant of the
  existing test_pipelock_blocks_secret_post. Posts a credential
  pattern in the body over HTTPS through the bottle. mitmproxy
  bumps the CONNECT (the agent trusts the per-bottle ephemeral CA
  installed by provision_ca), the addon forwards the decrypted
  request to pipelock, pipelock returns 403 with the known
  `blocked: ...` body shape, and the addon short-circuits the
  flow with status=403 + X-Pipelock-Bridge: block. The two-axis
  assertion (status + header) proves the addon-mediated path is
  what produced the block, not some other layer.

- test_mitmproxy_allows_normal_https: hits raw.githubusercontent.com
  (a baked-in allowlist host) over HTTPS through the bottle.
  Verifies the addon's allow path: mitmproxy bumps, addon
  forwards to pipelock for the scan, pipelock allows, mitmproxy
  proceeds to the real upstream, response comes back through. The
  absence of X-Pipelock-Bridge on the response is the signal that
  the addon didn't short-circuit. Body length sanity-checks that
  the response is real upstream content, not a synthesized stub.

Both probes are stdlib-only Node (http.request CONNECT + tls.connect
on the tunneled socket) — pulling in undici as a dep would be the
clean way to do HTTPS-through-proxy but is out of scope.

The earlier integration tests still pass with mitmproxy in path:
their assertions hold under the new topology, though their semantic
coverage shifts (e.g. test_pipelock_allow_node now exercises
mitmproxy's CONNECT-200 path rather than pipelock's host allowlist
on CONNECT). Updating those tests is a follow-up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis commented 2026-05-12 14:08:22 -04:00
Author
Owner
Copy Link
Copy Source

Closing this PR.

The whole design rests on a falsified premise. docs/research/pipelock-assessment.md says "Pipelock does not perform TLS inspection (no CA trust injection)" and docs/research/tls-mitm-for-pipelock.md inherits that claim — both wrong against pipelock v2.3.0. Pipelock has full tls_interception (enabled, ca_cert, ca_key, cert_ttl, passthrough_domains, ...) plus a pipelock tls init / install-ca / show-ca CLI. Empirical proof from an impl-walkthrough spike: pipelock generated a CA-signed leaf for httpbin.org (subject: CN=httpbin.org, issuer: O=Pipelock; CN=Pipelock CA), terminated the agent's TLS, and emitted:

"event":"anomaly","method":"CONNECT","scanner":"tls_intercept","reason":"TLS MITM interception active"
"event":"blocked","method":"POST","scanner":"body_dlp","reason":"request body contains secret: GitHub Token"

Standalone, no mitmproxy in the path.

This means mitmproxy is unnecessary and the entire mitmproxy → addon → pipelock topology in this PR is dead weight. Replaced by a fresh PRD that simply enables pipelock's native tls_interception block. Both research notes need a correction in the new PR so they stop misleading.

Follow-up PR: (pending).

Closing this PR. The whole design rests on a falsified premise. `docs/research/pipelock-assessment.md` says *"Pipelock does not perform TLS inspection (no CA trust injection)"* and `docs/research/tls-mitm-for-pipelock.md` inherits that claim — both wrong against pipelock v2.3.0. Pipelock has full `tls_interception` (`enabled`, `ca_cert`, `ca_key`, `cert_ttl`, `passthrough_domains`, ...) plus a `pipelock tls init` / `install-ca` / `show-ca` CLI. Empirical proof from an impl-walkthrough spike: pipelock generated a CA-signed leaf for httpbin.org (`subject: CN=httpbin.org, issuer: O=Pipelock; CN=Pipelock CA`), terminated the agent's TLS, and emitted: ``` "event":"anomaly","method":"CONNECT","scanner":"tls_intercept","reason":"TLS MITM interception active" "event":"blocked","method":"POST","scanner":"body_dlp","reason":"request body contains secret: GitHub Token" ``` Standalone, no mitmproxy in the path. This means mitmproxy is unnecessary and the entire `mitmproxy → addon → pipelock` topology in this PR is dead weight. Replaced by a fresh PRD that simply enables pipelock's native `tls_interception` block. Both research notes need a correction in the new PR so they stop misleading. Follow-up PR: (pending).
didericis closed this pull request 2026-05-12 14:08:23 -04:00
Some checks are pending
test / unit (pull_request) Successful in 20s
Details
test / integration (pull_request) Successful in 15s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
didericis-claude (didericis (claude)) didericis-codex (didericis (codex)) didericis-gemma (gemma) didericis
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: didericis/bot-bottle#8
Delete Branch "mitmproxy-tls-interception"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?