docs(research): add Betterleaks switching analysis
test / unit (pull_request) Successful in 13s
test / integration (pull_request) Successful in 28s

This commit is contained in:
2026-05-24 23:59:42 -04:00
parent c33930290f
commit 1f9722ae27
+70
View File
@@ -326,6 +326,76 @@ This puts the answer to PRD 0012's open question as:
(free-text justification), and as a versioned diff to the
baseline file (which is committed alongside the manifest).
## Should we switch to Betterleaks?
Not yet. Worth tracking. The wins are real on paper, but a
security gate is the wrong place to bet on a three-month-old
project, and the vendor-side benchmark numbers (98.6% recall vs
gitleaks's 70.4% on CredData) have not been independently
reproduced in published sources.
### What Betterleaks would add for claude-bottle
- **Detection coverage on encoded secrets.** Native handling of
doubly- and triply-encoded matches. This matters in the
threat model: an agent coaxed into base64-wrapping a token to
slip past the gate is a plausible attack, and gitleaks's
entropy-based approach misses many encodings.
- **CEL filters instead of the TOML allowlist struct.** More
expressive than `paths + regexes + condition`. Doesn't unlock
anything fundamental, but cleaner if exception rules ever need
conjunctive logic ("allow if path matches X *and* line contains
a documented placeholder string").
- **Agent-aware output.** Flag-based, low-token-overhead CLI
output designed for an AI agent (like one running inside a
bottle) to consume. Useful for the `/request-gate-exception`
slash command's parsing path; ergonomic win, not security-
load-bearing.
- **Avoids the frozen-upstream problem.** Gitleaks is feature-
complete, so a migration is eventually forced; the question is
whether to pay the cost now or later.
### What it would cost
- The existing pre-receive hook calls `gitleaks git
--log-opts=<range> --no-banner --redact`. Betterleaks's CLI
surface is similar but not identical and was not designed as a
drop-in for that specific invocation. Some hook rewrite is
likely.
- Whether Betterleaks has a baseline-file equivalent (the
storage format Option B recommends) is unconfirmed at the time
of writing. If it does not, Option B's storage format would
have to be re-derived against whatever Betterleaks offers.
- A three-month-old project has fewer security audits, fewer
third-party integrations, and a smaller community than
gitleaks has accumulated since 2018. The gate is exactly where
that asymmetry matters most.
### Criteria to revisit
Revisit when at least two of the following are true:
- Betterleaks has accumulated ~12 months of stable releases and
at least one external security audit.
- The CredData benchmark numbers have been independently
reproduced.
- A baseline-file equivalent (or a clearly better primitive for
per-finding approval storage) is shipped and documented.
- Gitleaks releases a security patch we cannot apply because the
underlying issue is a design choice rather than a bug — i.e.
the frozen status starts to bite.
### Forward-compatibility for the approval flow
Independent of the switching decision, Option B should treat the
choice of scanner as substitutable. Practically: the approval-
flow contract is "an approval is a finding-specific JSON record
stored alongside the manifest"; the *format* of that record
(gitleaks baseline schema today, something else later) is a
serialization concern downstream of the contract. Swapping
scanners then becomes a serialization migration, not a flow
redesign.
## Cross-references
- PRD 0008 — git-gate design and "no bypass" non-goal.