diff --git a/docs/research/dlp-alternatives-to-pipelock.md b/docs/research/dlp-alternatives-to-pipelock.md new file mode 100644 index 0000000..62f81af --- /dev/null +++ b/docs/research/dlp-alternatives-to-pipelock.md @@ -0,0 +1,179 @@ +# DLP alternatives to pipelock: per-route configuration and response handling + +## Question + +Pipelock lacks support for per-route or per-host response scanning rules, making it impossible to skip DLP scanning for large binary downloads (e.g., `.whl` files) while keeping scanning enabled for other traffic on the same host. Should we replace pipelock with a purpose-built DLP/token-scanning proxy that supports granular per-route configuration? + +## Summary + +Yes. Pipelock's flat, global configuration is fundamentally at odds with the per-route model bot-bottle is built on. A custom or configurable DLP proxy built atop mitmproxy (which we already use for egress) would let us: + +1. **Skip DLP scanning selectively** — e.g., scan responses from PyPI for credentials but skip scanning `.whl` file contents +2. **Configure scanning per-route** — different rules for different hosts/paths without global toggles +3. **Reduce operational surface** — one proxy (egress) instead of two (egress + pipelock) +4. **Target AI-specific threats** — focus on credential exfiltration and prompt injection instead of generic DLP + +**Tradeoff:** We'd need to maintain our own scanning logic. Pipelock provides out-of-the-box BIP-39 seed-phrase detection, entropy checks, and pluggable DLP rules. Building custom logic means we need to be explicit about what we're protecting against and keep that code auditable. + +## Current pipelock limitations + +### Issue 1: No per-route response scanning rules + +Pipelock's response scanning is part of TLS interception — a global feature with no per-host knobs: + +```yaml +tls_interception: + enabled: true + passthrough_domains: [...] # Can skip MITM, but not just response scanning +``` + +**Status:** Tested with pipelock v2.3.0. Confirmed that: +- `response_body_scanning` config field doesn't exist +- No way to set per-host response size limits +- No way to skip scanning for specific file extensions +- `tls_passthrough: true` disables both request AND response scanning (we want request scanning to stay on) + +### Issue 2: Global configuration only + +All of pipelock's scanning rules are global. If route A wants to skip `.whl` scanning and route B wants to skip `.tar.gz`, there's nowhere to express that distinction — the config is flat. + +### Issue 3: LLM prompt-specific false positives + +Pipelock's BIP-39 seed-phrase detector fires on any 12+ English words matching a checksum, which is common in LLM prompts/responses. Bot-bottle disables this detector globally, sacrificing protection. + +## Replacement design: mitmproxy-based DLP addon + +Since bot-bottle already uses mitmproxy for egress (PRD 0017), we can extend the mitmproxy addon to do DLP scanning alongside egress rules: + +### Architecture + +``` +Agent + ↓ (HTTP_PROXY=http://egress:8080) +Egress (mitmproxy) + ├─ Addon 1: Path allowlisting (current) + ├─ Addon 2: Credential injection (current) + └─ Addon 3: DLP scanning (NEW) + ├─ Config: per-route scanning rules from manifest + ├─ Detectors: token patterns, prompt injection, entropy + └─ Action: block/warn based on route config +``` + +### Per-route configuration in manifest + +```yaml +egress: + routes: + - host: api.anthropic.com + dlp: + enabled: true + detectors: [tokens, entropy] + + - host: files.pythonhosted.org + dlp: + enabled: true + request_only: true # Scan outbound, skip response + skip_extensions: [".whl", ".tar.gz"] + + - host: internal-service.corp + dlp: + enabled: false # Trusted internal, no scanning +``` + +### Detector design + +Three core detectors, each with tunable sensitivity: + +1. **Token detector** + - Regex patterns for API keys (AWS `AKIA`, GitHub `ghp_`, etc.) + - Anthropic/OpenAI API keys + - OAuth tokens (Bearer patterns) + - Action: Block immediately with no false-positive tolerance + +2. **Entropy detector** + - Shannon entropy threshold (bits/char) + - Flags high-entropy secrets (tunable per-route) + - Current pipelock default: 4.5 bits/char + - Action: Warn or block based on route config + +3. **Prompt injection detector** (phase 2) + - Detect attempts to exfiltrate system prompts via LLM outputs + - Pattern: responses containing "system prompt", "instructions", "directive" + credential + - Action: Block or sample for audit + +### Advantages over pipelock + +| Aspect | Pipelock | Mitmproxy addon | +|--------|----------|-----------------| +| Per-route rules | ❌ (global only) | ✅ (manifest-driven) | +| Response-specific config | ❌ (all-or-nothing) | ✅ (request_only, skip_extensions) | +| Request scanning overhead | ✅ (lightweight) | ~same | +| Maintenance burden | Low (third-party) | High (custom code) | +| Auditability | Closed source | ✅ (in-repo) | +| AI-specific detection | Limited | ✅ (token patterns, prompt injection) | +| Code reuse | None | ✅ (egress addon framework) | + +### Disadvantages + +1. **Maintenance responsibility** — We own the security logic. Any bugs in detector regexes or entropy thresholds are our problem. +2. **Feature parity gap** — Pipelock's BIP-39 detector is sophisticated. We'd need to decide: replicate it, skip it, or ship a simplified version. +3. **Performance** — Custom Python detectors will be slower than pipelock's Go implementation. Benchmarking needed. +4. **Coverage breadth** — Pipelock covers generic DLP (credit cards, SSNs, etc.). We'd focus narrowly on AI/credential exfil. + +## Alternative: Configurable pipelock fork + +Rather than build from scratch, fork pipelock and add `response_body_scanning` config: + +```yaml +response_body_scanning: + enabled: true + skip_extensions: [".whl", ".tar.gz"] + max_response_bytes: 104857600 # 100MB +``` + +**Pros:** +- Reuses existing detectors and maturity +- Lower maintenance burden +- Clear path to upstream (could be PR'd) + +**Cons:** +- Still maintains a fork +- Pipelock's maintainers may not want global per-host rules +- Go code is farther from our codebase (harder to audit) +- Doesn't solve prompt-injection detection + +## Recommendation + +**Build the mitmproxy addon** (phase 1: tokens + entropy; phase 2: prompt injection). + +**Rationale:** +1. Bot-bottle already owns the mitmproxy egress addon — extending it keeps security logic in-repo and auditable. +2. Per-route DLP configuration aligns with bot-bottle's design (PRD 0017 is already per-route). +3. Replacing pipelock reduces sidecar count and operational surface. +4. AI-specific detectors (tokens, prompt injection) matter more than generic DLP for agent containment. + +**Fallback:** If performance testing shows unacceptable latency in the Python addon, revisit the pipelock fork approach. + +## Implementation phases + +### Phase 1: MVP (2-3 weeks) +- Token detector (regex for API key patterns) +- Entropy detector (reuse pipelock thresholds) +- Per-route `dlp: {enabled, request_only, skip_extensions}` config +- Block on token match, warn on entropy hit + +### Phase 2: Prompt injection (1-2 weeks) +- Pattern detector for system prompt exfiltration +- Integrates with phase 1 config + +### Phase 3: Hardening (optional) +- Custom entropy heuristics for LLM payloads +- Sampling/audit mode for high-entropy responses +- Rate limiting on DLP blocks + +## Open questions + +1. **Performance:** How much latency does Python string-matching add? Benchmark against pipelock. +2. **False positives:** Will entropy detector trip on legitimate high-entropy traffic (e.g., binary API responses)? Need real-world testing. +3. **Coverage:** Are regex patterns sufficient, or do we need more sophisticated token detection (e.g., format validation)? +4. **Upstream:** If we build this, should we upstream it as an option to pipelock, or keep it bot-bottle-specific?