docs(research): add Betterleaks switching analysis
This commit is contained in:
@@ -326,6 +326,76 @@ This puts the answer to PRD 0012's open question as:
|
||||
(free-text justification), and as a versioned diff to the
|
||||
baseline file (which is committed alongside the manifest).
|
||||
|
||||
## Should we switch to Betterleaks?
|
||||
|
||||
Not yet. Worth tracking. The wins are real on paper, but a
|
||||
security gate is the wrong place to bet on a three-month-old
|
||||
project, and the vendor-side benchmark numbers (98.6% recall vs
|
||||
gitleaks's 70.4% on CredData) have not been independently
|
||||
reproduced in published sources.
|
||||
|
||||
### What Betterleaks would add for claude-bottle
|
||||
|
||||
- **Detection coverage on encoded secrets.** Native handling of
|
||||
doubly- and triply-encoded matches. This matters in the
|
||||
threat model: an agent coaxed into base64-wrapping a token to
|
||||
slip past the gate is a plausible attack, and gitleaks's
|
||||
entropy-based approach misses many encodings.
|
||||
- **CEL filters instead of the TOML allowlist struct.** More
|
||||
expressive than `paths + regexes + condition`. Doesn't unlock
|
||||
anything fundamental, but cleaner if exception rules ever need
|
||||
conjunctive logic ("allow if path matches X *and* line contains
|
||||
a documented placeholder string").
|
||||
- **Agent-aware output.** Flag-based, low-token-overhead CLI
|
||||
output designed for an AI agent (like one running inside a
|
||||
bottle) to consume. Useful for the `/request-gate-exception`
|
||||
slash command's parsing path; ergonomic win, not security-
|
||||
load-bearing.
|
||||
- **Avoids the frozen-upstream problem.** Gitleaks is feature-
|
||||
complete, so a migration is eventually forced; the question is
|
||||
whether to pay the cost now or later.
|
||||
|
||||
### What it would cost
|
||||
|
||||
- The existing pre-receive hook calls `gitleaks git
|
||||
--log-opts=<range> --no-banner --redact`. Betterleaks's CLI
|
||||
surface is similar but not identical and was not designed as a
|
||||
drop-in for that specific invocation. Some hook rewrite is
|
||||
likely.
|
||||
- Whether Betterleaks has a baseline-file equivalent (the
|
||||
storage format Option B recommends) is unconfirmed at the time
|
||||
of writing. If it does not, Option B's storage format would
|
||||
have to be re-derived against whatever Betterleaks offers.
|
||||
- A three-month-old project has fewer security audits, fewer
|
||||
third-party integrations, and a smaller community than
|
||||
gitleaks has accumulated since 2018. The gate is exactly where
|
||||
that asymmetry matters most.
|
||||
|
||||
### Criteria to revisit
|
||||
|
||||
Revisit when at least two of the following are true:
|
||||
|
||||
- Betterleaks has accumulated ~12 months of stable releases and
|
||||
at least one external security audit.
|
||||
- The CredData benchmark numbers have been independently
|
||||
reproduced.
|
||||
- A baseline-file equivalent (or a clearly better primitive for
|
||||
per-finding approval storage) is shipped and documented.
|
||||
- Gitleaks releases a security patch we cannot apply because the
|
||||
underlying issue is a design choice rather than a bug — i.e.
|
||||
the frozen status starts to bite.
|
||||
|
||||
### Forward-compatibility for the approval flow
|
||||
|
||||
Independent of the switching decision, Option B should treat the
|
||||
choice of scanner as substitutable. Practically: the approval-
|
||||
flow contract is "an approval is a finding-specific JSON record
|
||||
stored alongside the manifest"; the *format* of that record
|
||||
(gitleaks baseline schema today, something else later) is a
|
||||
serialization concern downstream of the contract. Swapping
|
||||
scanners then becomes a serialization migration, not a flow
|
||||
redesign.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- PRD 0008 — git-gate design and "no bypass" non-goal.
|
||||
|
||||
Reference in New Issue
Block a user