From aa44feea02889a74a92b1773b1a00db524da29ca Mon Sep 17 00:00:00 2001 From: didericis Date: Sat, 18 Jul 2026 07:00:22 -0400 Subject: [PATCH] docs(research): note on malicious-commit scanning at the git-gate + paid-feature analysis MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a research note on whether/how to scan for malicious code (not just secrets) in commits pushed through the git-gate, and whether the semantic (LLM) layer is a defensible paid feature. Verdict: no scanner reliably detects malicious code (undecidable + adversarial), so the frame is raise-cost + cover-the-obvious + human-gate the dangerous. Ranked layers: dependency/supply-chain scanning (Socket/OSV/ GuardDog) > heuristic/obfuscation (Semgrep-on-diff) > risk-based human gating via the existing supervise plane > best-effort LLM diff-review. Fast scanners inline in the synchronous pre-receive; heavy analysis async. Monetization: the paid unit is the governed git-egress review bundle (managed semantic review + web-console human-review flow + RBAC + audit + cross-run policy), not the raw scanner — which stays OSS like gitleaks. Extends the egress audit+custody wedge to code artifacts; the supervise console generalizes across all proposal types (egress, gitleaks, commit review). Sell the workflow, not the detector's accuracy. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD --- docs/research/malicious-commit-scanning.md | 204 +++++++++++++++++++++ 1 file changed, 204 insertions(+) create mode 100644 docs/research/malicious-commit-scanning.md diff --git a/docs/research/malicious-commit-scanning.md b/docs/research/malicious-commit-scanning.md new file mode 100644 index 0000000..cac426f --- /dev/null +++ b/docs/research/malicious-commit-scanning.md @@ -0,0 +1,204 @@ +# Scanning for malicious commits at the git-gate — and whether the semantic layer is a paid feature + +A sandboxed agent's most durable way out isn't live exfiltration — it's the +**artifact it pushes**. Code committed through the git-gate leaves the +sandbox and runs *later*, in CI or production, with none of the bottle's +isolation or egress control. gitleaks covers *secrets* on that channel +(see [`git-gate` pre-receive](../../bot_bottle/git_gate_render.py)); this +note asks the harder question — can we scan for *malicious code* in +commits — and the product question the answer raises: is the semantic +(LLM) layer a decent paid feature on "git egress," alongside a paid +human-review console for the supervise flow? + +## Summary + +- **No scanner reliably detects malicious code.** It's undecidable in + general and adversarial in practice — a capable agent hides a backdoor + from any pattern scanner (and from human reviewers). So the frame is not + "detect malice"; it's **raise the cost, cover the common/obvious cases, + and put a human on the high-leverage ones.** +- **Layered, ranked by ROI:** (1) supply-chain / malicious-dependency + scanning (Socket, OSV-Scanner, GuardDog) — highest ROI, best git fit; + (2) fast heuristic / obfuscation / dangerous-sink scanning + (Semgrep-on-diff) — gitleaks-shaped; (3) **risk-based human gating** via + the existing supervise plane on high-risk file classes; (4) **LLM + semantic diff-review** as a best-effort backstop for novel backdoors. +- **Architecture:** the git-gate's `pre-receive` is *synchronous* (it + blocks the push), so run only fast scanners inline and push heavy + analysis (CodeQL, full LLM review) to an **async post-push CI lane** that + can't hold the developer hostage. +- **Monetization verdict: yes — but the paid unit is the *governed + git-egress review bundle*, not the raw scanner.** The pre-receive + mechanics and the fast OSS scanners stay free (open-core social + contract, same as gitleaks). The sell is the control-plane workflow + around them: **managed semantic review + a web-console human-review flow + + RBAC + audit retention + cross-run policy.** A bare per-push scan is + borderline "single-run = free" by our own heuristic, and its detection + accuracy is not something to sell on. Sell the *workflow and the + governance*, not the detector. + +## The threat: temporal escape + +This is the one containment fundamentally can't close. bot-bottle's VM +isolation + egress DLP shrink *live* exfiltration, but the git push +channel launders a payload past all of it: the agent commits code that +looks benign at review time and does its work later, outside the +perimeter — the "blind-XSS"-style temporal escape that both the "3 reasons +sandboxing won't secure your AI agent" talk and Embroidery's "In sandboxes +we (shouldn't) trust" call the sharpest gap. gitleaks gates *secrets* on +this channel; malicious *code* is the uncovered surface, and it's the +behavioral half of the isolation-vs-behavioral split noted in +[`agent-sandbox-landscape.md`](agent-sandbox-landscape.md). + +## Why it's fundamentally hard + +Distinguishing malicious from legitimate code is undecidable in general +and adversarial in the specific. The underhanded-code contests show humans +hide backdoors from *human reviewers*; a scanner reduces to regex/dataflow +heuristics a motivated agent writes around. So "catch malicious commits" +is the wrong success metric. The right ones: *how much did we raise the +cost*, *did we cover the common and the obvious*, and *did a human see the +dangerous changes before they left*. + +## Landscape of git-pluggable approaches (ranked by ROI) + +1. **Supply-chain / malicious-dependency scanning — highest ROI.** The + most likely agent-introduced threat is a malicious or typosquatted + *dependency*, not a hand-written backdoor — and it's far more + detectable. **Socket** (behavioral analysis of packages: install + scripts, obfuscation, unexpected network/fs access, typosquats) is the + closest thing to malicious-intent detection that actually works; + **OSV-Scanner** (known-bad DB, fast) and **GuardDog** (PyPI/npm + heuristics) complement it. Trigger only when a commit changes a + lockfile/manifest → block or escalate. +2. **Fast heuristic / obfuscation / dangerous-sink scanning.** **Semgrep** + with a curated ruleset, scanning the *diff* (fast enough for + pre-receive). Catches obfuscation (`eval(atob(...))`, base64/hex blobs, + minified source), exfil primitives (`curl | sh`, reverse shells, + DNS-exfil), and dangerous sinks (`shell=True`, `pickle.loads`, unsafe + `yaml.load`, `child_process.exec`). Tune noisy rules to *escalate*, not + block. +3. **Risk-based human gating — the most robust control, already + half-built.** The git-gate already escalates `# gitleaks:allow` + findings to a supervisor proposal and blocks the push pending approval + (`supervise_gitleaks_allow`, [`git_gate_render.py`](../../bot_bottle/git_gate_render.py)). + Extend the same flow to **high-risk file classes**: any commit touching + CI/build/deploy scripts, auth/crypto code, egress config, or + adding/changing dependencies → route to `./cli.py supervise`. This is + attribution/policy, not detection, and it's the strongest thing here — + a human on exactly the temporal-escape surfaces. +4. **LLM semantic diff-review — the behavioral backstop.** The only + approach with a shot at a novel, benign-looking backdoor: a model + reviews the diff for a backdoor / exfiltration / obfuscated payload / + dangerous behavior. This is the behavioral-detection thesis applied to + the *artifact* (the diff) rather than the agent's reasoning. Real + caveats: false positives and negatives, latency/cost, it can be + prompt-injected by hostile content *in the diff itself*, and it invites + over-trust. Useful, never a guarantee. + +**Not for this job:** CodeQL, Trivy, Grype, Bandit. They find *known +vulns and insecure patterns* (bugs), not deliberate backdoors, and the +powerful ones (CodeQL taint) need a build + database — too heavy for a +synchronous gate. They belong in the async CI lane if at all. + +## Fit into bot-bottle's git-gate + +The `pre-receive` hook today is: gitleaks-scan each ref → escalate +`# gitleaks:allow` findings to supervise → forward to upstream +([`git_gate_render.py`](../../bot_bottle/git_gate_render.py)). The +additions slot in cleanly: + +- **Inline (fast), before forward:** a dep-scan phase (on manifest/lockfile + change) and a Semgrep-diff phase. Findings block or open a supervise + proposal, same shape as gitleaks. +- **New supervise tool types** alongside the existing + `egress-block/allow`, `gitleaks-allow`, `egress-token-allow` + ([`supervise_types.py`](../../bot_bottle/supervise_types.py)) — e.g. a + `commit-review` proposal for risky-file-class gating and for semantic + review. The supervise plane is already the right abstraction; this is + another *producer* feeding it, and [`supervise_server.py`](../../bot_bottle/supervise_server.py) + (JSON-RPC) is already the console backend. +- **Async lane (heavy):** full LLM review + any CodeQL run out of band + after the push, feeding the same review/audit surface, so the + synchronous gate stays fast. + +## The product question: paid feature on git egress? + +Restating the open-core line bot-bottle runs on: *give away the +sandbox/runtime, charge for the control plane; single-run/single-node = +free, cross-run aggregation + central enforcement + identity/fleet = paid; +the moat is uniform egress audit + secret custody + policy across +untrusted agents.* + +Against that line, the split is clean: + +**Free (OSS runtime — the trust funnel):** +- the `pre-receive` gate mechanics and gitleaks; +- wiring the OSS scanners (Socket CLI / OSV-Scanner / Semgrep); +- the CLI supervise flow. +Keeping the raw scanners free is the same social contract as gitleaks and +preserves the bottom-up distribution funnel. + +**Paid (the governed git-egress bundle — the control plane):** +- **Managed semantic diff-review** — hosted inference + a curated, + maintained malicious-pattern/policy set. This is *capability* (metered), + not *insurance* — the thing individuals actually pay for. Position it as + **governed code-egress review**, not "we resell inference" (the + monetization notes explicitly warn against reselling compute). +- **The web-console supervise/review flow — the strongest anchor.** Turn + the CLI `./cli.py supervise` approval into a real review surface: + rendered diff + finding context, approve/reject, **who-approved audit + trail, RBAC on approvers, mobile/phone-control** (ties to the + dashboard/vault north star). This is "central enforcement + + identity/fleet = paid" almost verbatim — and it generalizes across + *every* supervise proposal (egress block/allow, gitleaks-allow, + commit-review), so it's worth building for the whole plane, with the + semantic check as one producer. +- **Cross-run governance:** fleet-wide policy for what escalates, + review-decision history/search/export, and drift alerts. + +**Why it fits the moat rather than bolting on:** a git push *is* an egress +channel. A semantic review + human approval + audit on it extends the +uniform "egress audit + custody + policy across untrusted agents" wedge to +**code artifacts** — the same product, applied to the one channel gitleaks +only half-covers. That's on-moat, not a detour. + +**The honest nuance (don't oversell):** a bare per-push LLM scan is +arguably *free* by the single-run heuristic, and its detection accuracy is +not defensible to charge for. The paid value is the **governance around +it** — the console, RBAC, audit retention, cross-run policy — plus the +managed capability. Sell the *review-and-approve-and-audit workflow*; let +the detector be explicitly best-effort. And per the monetization +guardrail, the "anti-corporate" free crowd must not veto these team +features: the review console + RBAC + audit *are* the monetization. + +## Recommendation + +1. **Land the free layer first.** Add the dep-scan and Semgrep-diff phases + to `pre-receive`, and extend supervise to risky-file-class gating — + reuses existing machinery, immediate value, stays OSS. +2. **Build the supervise web console** over `supervise_server`'s JSON-RPC + (already the Phase-1 move in the monetization path). This is the paid + anchor and it serves *all* proposal types, not just commit review. +3. **Add managed semantic diff-review as a paid producer** feeding that + console — "governed code-egress review," metered, explicitly + best-effort on detection. +4. **Don't oversell detection.** Market the workflow (review + approve + + audit) and the cross-run policy/RBAC, where the value is real and + defensible; keep the raw scanners open. + +## Sources / references + +- [`agent-sandbox-landscape.md`](agent-sandbox-landscape.md) — the + egress-DLP gap and isolation-vs-behavioral framing. +- Git-gate internals: [`git_gate_render.py`](../../bot_bottle/git_gate_render.py), + [`supervise_types.py`](../../bot_bottle/supervise_types.py), + [`supervise_server.py`](../../bot_bottle/supervise_server.py). +- External tools: Socket (socket.dev), OSV-Scanner (google/osv-scanner), + GuardDog (DataDog/guarddog), Semgrep (semgrep/semgrep). +- Threat framing: "3 reasons sandboxing won't secure your AI agent" + (youtube TsYDazwHJ6U); Embroidery, "In sandboxes we (shouldn't) trust." +- The authoritative monetization/positioning analysis (the open-core line, + the wedge, single-run-free/cross-run-paid) lives in the **separate + `bot-bottle-console` repo**, not this one — cited here from memory, not + linked.