bot-bottle/docs/research/malicious-commit-scanning.md

Scanning for malicious commits at the git-gate — and whether the semantic layer is a paid feature

A sandboxed agent's most durable way out isn't live exfiltration — it's the artifact it pushes. Code committed through the git-gate leaves the sandbox and runs later, in CI or production, with none of the bottle's isolation or egress control. gitleaks covers secrets on that channel (see git-gate pre-receive); this note asks the harder question — can we scan for malicious code in commits — and the product question the answer raises: is the semantic (LLM) layer a decent paid feature on "git egress," alongside a paid human-review console for the supervise flow?

Summary

• No scanner reliably detects malicious code. It's undecidable in general and adversarial in practice — a capable agent hides a backdoor from any pattern scanner (and from human reviewers). So the frame is not "detect malice"; it's raise the cost, cover the common/obvious cases, and put a human on the high-leverage ones.
• Layered, ranked by ROI: (1) supply-chain / malicious-dependency scanning (Socket, OSV-Scanner, GuardDog) — highest ROI, best git fit; (2) fast heuristic / obfuscation / dangerous-sink scanning (Semgrep-on-diff) — gitleaks-shaped; (3) risk-based human gating via the existing supervise plane on high-risk file classes; (4) LLM semantic diff-review as a best-effort backstop for novel backdoors.
• Architecture: the git-gate's pre-receive is synchronous (it blocks the push), so run only fast scanners inline and push heavy analysis (CodeQL, full LLM review) to an async post-push CI lane that can't hold the developer hostage.
• Monetization verdict: yes — but the paid unit is the governed git-egress review bundle, not the raw scanner. The pre-receive mechanics and the fast OSS scanners stay free (open-core social contract, same as gitleaks). The sell is the control-plane workflow around them: **managed semantic review + a web-console human-review flow
  • RBAC + audit retention + cross-run policy.** A bare per-push scan is borderline "single-run = free" by our own heuristic, and its detection accuracy is not something to sell on. Sell the workflow and the governance, not the detector.

The threat: temporal escape

This is the one containment fundamentally can't close. bot-bottle's VM isolation + egress DLP shrink live exfiltration, but the git push channel launders a payload past all of it: the agent commits code that looks benign at review time and does its work later, outside the perimeter — the "blind-XSS"-style temporal escape that both the "3 reasons sandboxing won't secure your AI agent" talk and Embroidery's "In sandboxes we (shouldn't) trust" call the sharpest gap. gitleaks gates secrets on this channel; malicious code is the uncovered surface, and it's the behavioral half of the isolation-vs-behavioral split noted in agent-sandbox-landscape.md.

Why it's fundamentally hard

Distinguishing malicious from legitimate code is undecidable in general and adversarial in the specific. The underhanded-code contests show humans hide backdoors from human reviewers; a scanner reduces to regex/dataflow heuristics a motivated agent writes around. So "catch malicious commits" is the wrong success metric. The right ones: how much did we raise the cost, did we cover the common and the obvious, and did a human see the dangerous changes before they left.

Landscape of git-pluggable approaches (ranked by ROI)

1. Supply-chain / malicious-dependency scanning — highest ROI. The most likely agent-introduced threat is a malicious or typosquatted dependency, not a hand-written backdoor — and it's far more detectable. Socket (behavioral analysis of packages: install scripts, obfuscation, unexpected network/fs access, typosquats) is the closest thing to malicious-intent detection that actually works; OSV-Scanner (known-bad DB, fast) and GuardDog (PyPI/npm heuristics) complement it. Trigger only when a commit changes a lockfile/manifest → block or escalate.
2. Fast heuristic / obfuscation / dangerous-sink scanning. Semgrep with a curated ruleset, scanning the diff (fast enough for pre-receive). Catches obfuscation (eval(atob(...)), base64/hex blobs, minified source), exfil primitives (curl | sh, reverse shells, DNS-exfil), and dangerous sinks (shell=True, pickle.loads, unsafe yaml.load, child_process.exec). Tune noisy rules to escalate, not block.
3. Risk-based human gating — the most robust control, already half-built. The git-gate already escalates # gitleaks:allow findings to a supervisor proposal and blocks the push pending approval (supervise_gitleaks_allow, git_gate_render.py). Extend the same flow to high-risk file classes: any commit touching CI/build/deploy scripts, auth/crypto code, egress config, or adding/changing dependencies → route to ./cli.py supervise. This is attribution/policy, not detection, and it's the strongest thing here — a human on exactly the temporal-escape surfaces.
4. LLM semantic diff-review — the behavioral backstop. The only approach with a shot at a novel, benign-looking backdoor: a model reviews the diff for a backdoor / exfiltration / obfuscated payload / dangerous behavior. This is the behavioral-detection thesis applied to the artifact (the diff) rather than the agent's reasoning. Real caveats: false positives and negatives, latency/cost, it can be prompt-injected by hostile content in the diff itself, and it invites over-trust. Useful, never a guarantee.

Not for this job: CodeQL, Trivy, Grype, Bandit. They find known vulns and insecure patterns (bugs), not deliberate backdoors, and the powerful ones (CodeQL taint) need a build + database — too heavy for a synchronous gate. They belong in the async CI lane if at all.

Fit into bot-bottle's git-gate

The pre-receive hook today is: gitleaks-scan each ref → escalate # gitleaks:allow findings to supervise → forward to upstream (git_gate_render.py). The additions slot in cleanly:

• Inline (fast), before forward: a dep-scan phase (on manifest/lockfile change) and a Semgrep-diff phase. Findings block or open a supervise proposal, same shape as gitleaks.
• New supervise tool types alongside the existing egress-block/allow, gitleaks-allow, egress-token-allow (supervise_types.py) — e.g. a commit-review proposal for risky-file-class gating and for semantic review. The supervise plane is already the right abstraction; this is another producer feeding it, and supervise_server.py (JSON-RPC) is already the console backend.
• Async lane (heavy): full LLM review + any CodeQL run out of band after the push, feeding the same review/audit surface, so the synchronous gate stays fast.

The product question: paid feature on git egress?

Restating the open-core line bot-bottle runs on: give away the sandbox/runtime, charge for the control plane; single-run/single-node = free, cross-run aggregation + central enforcement + identity/fleet = paid; the moat is uniform egress audit + secret custody + policy across untrusted agents.

Against that line, the split is clean:

Free (OSS runtime — the trust funnel):

• the pre-receive gate mechanics and gitleaks;
• wiring the OSS scanners (Socket CLI / OSV-Scanner / Semgrep);
• the CLI supervise flow. Keeping the raw scanners free is the same social contract as gitleaks and preserves the bottom-up distribution funnel.

Paid (the governed git-egress bundle — the control plane):

• Managed semantic diff-review — hosted inference + a curated, maintained malicious-pattern/policy set. This is capability (metered), not insurance — the thing individuals actually pay for. Position it as governed code-egress review, not "we resell inference" (the monetization notes explicitly warn against reselling compute).
• The web-console supervise/review flow — the strongest anchor. Turn the CLI ./cli.py supervise approval into a real review surface: rendered diff + finding context, approve/reject, who-approved audit trail, RBAC on approvers, mobile/phone-control (ties to the dashboard/vault north star). This is "central enforcement + identity/fleet = paid" almost verbatim — and it generalizes across every supervise proposal (egress block/allow, gitleaks-allow, commit-review), so it's worth building for the whole plane, with the semantic check as one producer.
• Cross-run governance: fleet-wide policy for what escalates, review-decision history/search/export, and drift alerts.

Why it fits the moat rather than bolting on: a git push is an egress channel. A semantic review + human approval + audit on it extends the uniform "egress audit + custody + policy across untrusted agents" wedge to code artifacts — the same product, applied to the one channel gitleaks only half-covers. That's on-moat, not a detour.

The honest nuance (don't oversell): a bare per-push LLM scan is arguably free by the single-run heuristic, and its detection accuracy is not defensible to charge for. The paid value is the governance around it — the console, RBAC, audit retention, cross-run policy — plus the managed capability. Sell the review-and-approve-and-audit workflow; let the detector be explicitly best-effort. And per the monetization guardrail, the "anti-corporate" free crowd must not veto these team features: the review console + RBAC + audit are the monetization.

Recommendation

1. Land the free layer first. Add the dep-scan and Semgrep-diff phases to pre-receive, and extend supervise to risky-file-class gating — reuses existing machinery, immediate value, stays OSS.
2. Build the supervise web console over supervise_server's JSON-RPC (already the Phase-1 move in the monetization path). This is the paid anchor and it serves all proposal types, not just commit review.
3. Add managed semantic diff-review as a paid producer feeding that console — "governed code-egress review," metered, explicitly best-effort on detection.
4. Don't oversell detection. Market the workflow (review + approve + audit) and the cross-run policy/RBAC, where the value is real and defensible; keep the raw scanners open.

Sources / references

• agent-sandbox-landscape.md — the egress-DLP gap and isolation-vs-behavioral framing.
• Git-gate internals: git_gate_render.py, supervise_types.py, supervise_server.py.
• External tools: Socket (socket.dev), OSV-Scanner (google/osv-scanner), GuardDog (DataDog/guarddog), Semgrep (semgrep/semgrep).
• Threat framing: "3 reasons sandboxing won't secure your AI agent" (youtube TsYDazwHJ6U); Embroidery, "In sandboxes we (shouldn't) trust."
• The authoritative monetization/positioning analysis (the open-core line, the wedge, single-run-free/cross-run-paid) lives in the separate bot-bottle-console repo, not this one — cited here from memory, not linked.