bot-bottle/docs/research/malicious-commit-scanning.md

# Scanning for malicious commits at the git-gate — and whether the semantic layer is a paid feature

A sandboxed agent's most durable way out isn't live exfiltration — it's the
**artifact it pushes**. Code committed through the git-gate leaves the
sandbox and runs *later*, in CI or production, with none of the bottle's
isolation or egress control. gitleaks covers *secrets* on that channel
(see [`git-gate` pre-receive](../../bot_bottle/git_gate_render.py)); this
note asks the harder question — can we scan for *malicious code* in
commits — and the product question the answer raises: is the semantic
(LLM) layer a decent paid feature on "git egress," alongside a paid
human-review console for the supervise flow?

## Summary

- **No scanner reliably detects malicious code.** It's undecidable in
  general and adversarial in practice — a capable agent hides a backdoor
  from any pattern scanner (and from human reviewers). So the frame is not
  "detect malice"; it's **raise the cost, cover the common/obvious cases,
  and put a human on the high-leverage ones.**
- **Layered, ranked by ROI:** (1) supply-chain / malicious-dependency
  scanning (Socket, OSV-Scanner, GuardDog) — highest ROI, best git fit;
  (2) fast heuristic / obfuscation / dangerous-sink scanning
  (Semgrep-on-diff) — gitleaks-shaped; (3) **risk-based human gating** via
  the existing supervise plane on high-risk file classes; (4) **LLM
  semantic diff-review** as a best-effort backstop for novel backdoors.
- **Architecture:** the git-gate's `pre-receive` is *synchronous* (it
  blocks the push), so run only fast scanners inline and push heavy
  analysis (CodeQL, full LLM review) to an **async post-push CI lane** that
  can't hold the developer hostage.
- **Monetization verdict: yes — but the paid unit is the *governed
  git-egress review bundle*, not the raw scanner.** The pre-receive
  mechanics and the fast OSS scanners stay free (open-core social
  contract, same as gitleaks). The sell is the control-plane workflow
  around them: **managed semantic review + a web-console human-review flow
  + RBAC + audit retention + cross-run policy.** A bare per-push scan is
  borderline "single-run = free" by our own heuristic, and its detection
  accuracy is not something to sell on. Sell the *workflow and the
  governance*, not the detector.

## The threat: temporal escape

This is the one containment fundamentally can't close. bot-bottle's VM
isolation + egress DLP shrink *live* exfiltration, but the git push
channel launders a payload past all of it: the agent commits code that
looks benign at review time and does its work later, outside the
perimeter — the "blind-XSS"-style temporal escape that both the "3 reasons
sandboxing won't secure your AI agent" talk and Embroidery's "In sandboxes
we (shouldn't) trust" call the sharpest gap. gitleaks gates *secrets* on
this channel; malicious *code* is the uncovered surface, and it's the
behavioral half of the isolation-vs-behavioral split noted in
[`agent-sandbox-landscape.md`](agent-sandbox-landscape.md).

## Why it's fundamentally hard

Distinguishing malicious from legitimate code is undecidable in general
and adversarial in the specific. The underhanded-code contests show humans
hide backdoors from *human reviewers*; a scanner reduces to regex/dataflow
heuristics a motivated agent writes around. So "catch malicious commits"
is the wrong success metric. The right ones: *how much did we raise the
cost*, *did we cover the common and the obvious*, and *did a human see the
dangerous changes before they left*.

## Landscape of git-pluggable approaches (ranked by ROI)

1. **Supply-chain / malicious-dependency scanning — highest ROI.** The
   most likely agent-introduced threat is a malicious or typosquatted
   *dependency*, not a hand-written backdoor — and it's far more
   detectable. **Socket** (behavioral analysis of packages: install
   scripts, obfuscation, unexpected network/fs access, typosquats) is the
   closest thing to malicious-intent detection that actually works;
   **OSV-Scanner** (known-bad DB, fast) and **GuardDog** (PyPI/npm
   heuristics) complement it. Trigger only when a commit changes a
   lockfile/manifest → block or escalate.
2. **Fast heuristic / obfuscation / dangerous-sink scanning.** **Semgrep**
   with a curated ruleset, scanning the *diff* (fast enough for
   pre-receive). Catches obfuscation (`eval(atob(...))`, base64/hex blobs,
   minified source), exfil primitives (`curl | sh`, reverse shells,
   DNS-exfil), and dangerous sinks (`shell=True`, `pickle.loads`, unsafe
   `yaml.load`, `child_process.exec`). Tune noisy rules to *escalate*, not
   block.
3. **Risk-based human gating — the most robust control, already
   half-built.** The git-gate already escalates `# gitleaks:allow`
   findings to a supervisor proposal and blocks the push pending approval
   (`supervise_gitleaks_allow`, [`git_gate_render.py`](../../bot_bottle/git_gate_render.py)).
   Extend the same flow to **high-risk file classes**: any commit touching
   CI/build/deploy scripts, auth/crypto code, egress config, or
   adding/changing dependencies → route to `./cli.py supervise`. This is
   attribution/policy, not detection, and it's the strongest thing here —
   a human on exactly the temporal-escape surfaces.
4. **LLM semantic diff-review — the behavioral backstop.** The only
   approach with a shot at a novel, benign-looking backdoor: a model
   reviews the diff for a backdoor / exfiltration / obfuscated payload /
   dangerous behavior. This is the behavioral-detection thesis applied to
   the *artifact* (the diff) rather than the agent's reasoning. Real
   caveats: false positives and negatives, latency/cost, it can be
   prompt-injected by hostile content *in the diff itself*, and it invites
   over-trust. Useful, never a guarantee.

**Not for this job:** CodeQL, Trivy, Grype, Bandit. They find *known
vulns and insecure patterns* (bugs), not deliberate backdoors, and the
powerful ones (CodeQL taint) need a build + database — too heavy for a
synchronous gate. They belong in the async CI lane if at all.

## Fit into bot-bottle's git-gate

The `pre-receive` hook today is: gitleaks-scan each ref → escalate
`# gitleaks:allow` findings to supervise → forward to upstream
([`git_gate_render.py`](../../bot_bottle/git_gate_render.py)). The
additions slot in cleanly:

- **Inline (fast), before forward:** a dep-scan phase (on manifest/lockfile
  change) and a Semgrep-diff phase. Findings block or open a supervise
  proposal, same shape as gitleaks.
- **New supervise tool types** alongside the existing
  `egress-block/allow`, `gitleaks-allow`, `egress-token-allow`
  ([`supervise_types.py`](../../bot_bottle/supervise_types.py)) — e.g. a
  `commit-review` proposal for risky-file-class gating and for semantic
  review. The supervise plane is already the right abstraction; this is
  another *producer* feeding it, and [`supervise_server.py`](../../bot_bottle/supervise_server.py)
  (JSON-RPC) is already the console backend.
- **Async lane (heavy):** full LLM review + any CodeQL run out of band
  after the push, feeding the same review/audit surface, so the
  synchronous gate stays fast.

## The product question: paid feature on git egress?

Restating the open-core line bot-bottle runs on: *give away the
sandbox/runtime, charge for the control plane; single-run/single-node =
free, cross-run aggregation + central enforcement + identity/fleet = paid;
the moat is uniform egress audit + secret custody + policy across
untrusted agents.*

Against that line, the split is clean:

**Free (OSS runtime — the trust funnel):**
- the `pre-receive` gate mechanics and gitleaks;
- wiring the OSS scanners (Socket CLI / OSV-Scanner / Semgrep);
- the CLI supervise flow.
Keeping the raw scanners free is the same social contract as gitleaks and
preserves the bottom-up distribution funnel.

**Paid (the governed git-egress bundle — the control plane):**
- **Managed semantic diff-review** — hosted inference + a curated,
  maintained malicious-pattern/policy set. This is *capability* (metered),
  not *insurance* — the thing individuals actually pay for. Position it as
  **governed code-egress review**, not "we resell inference" (the
  monetization notes explicitly warn against reselling compute).
- **The web-console supervise/review flow — the strongest anchor.** Turn
  the CLI `./cli.py supervise` approval into a real review surface:
  rendered diff + finding context, approve/reject, **who-approved audit
  trail, RBAC on approvers, mobile/phone-control** (ties to the
  dashboard/vault north star). This is "central enforcement +
  identity/fleet = paid" almost verbatim — and it generalizes across
  *every* supervise proposal (egress block/allow, gitleaks-allow,
  commit-review), so it's worth building for the whole plane, with the
  semantic check as one producer.
- **Cross-run governance:** fleet-wide policy for what escalates,
  review-decision history/search/export, and drift alerts.

**Why it fits the moat rather than bolting on:** a git push *is* an egress
channel. A semantic review + human approval + audit on it extends the
uniform "egress audit + custody + policy across untrusted agents" wedge to
**code artifacts** — the same product, applied to the one channel gitleaks
only half-covers. That's on-moat, not a detour.

**The honest nuance (don't oversell):** a bare per-push LLM scan is
arguably *free* by the single-run heuristic, and its detection accuracy is
not defensible to charge for. The paid value is the **governance around
it** — the console, RBAC, audit retention, cross-run policy — plus the
managed capability. Sell the *review-and-approve-and-audit workflow*; let
the detector be explicitly best-effort. And per the monetization
guardrail, the "anti-corporate" free crowd must not veto these team
features: the review console + RBAC + audit *are* the monetization.

## Recommendation

1. **Land the free layer first.** Add the dep-scan and Semgrep-diff phases
   to `pre-receive`, and extend supervise to risky-file-class gating —
   reuses existing machinery, immediate value, stays OSS.
2. **Build the supervise web console** over `supervise_server`'s JSON-RPC
   (already the Phase-1 move in the monetization path). This is the paid
   anchor and it serves *all* proposal types, not just commit review.
3. **Add managed semantic diff-review as a paid producer** feeding that
   console — "governed code-egress review," metered, explicitly
   best-effort on detection.
4. **Don't oversell detection.** Market the workflow (review + approve +
   audit) and the cross-run policy/RBAC, where the value is real and
   defensible; keep the raw scanners open.

## Sources / references

- [`agent-sandbox-landscape.md`](agent-sandbox-landscape.md) — the
  egress-DLP gap and isolation-vs-behavioral framing.
- Git-gate internals: [`git_gate_render.py`](../../bot_bottle/git_gate_render.py),
  [`supervise_types.py`](../../bot_bottle/supervise_types.py),
  [`supervise_server.py`](../../bot_bottle/supervise_server.py).
- External tools: Socket (socket.dev), OSV-Scanner (google/osv-scanner),
  GuardDog (DataDog/guarddog), Semgrep (semgrep/semgrep).
- Threat framing: "3 reasons sandboxing won't secure your AI agent"
  (youtube TsYDazwHJ6U); Embroidery, "In sandboxes we (shouldn't) trust."
- The authoritative monetization/positioning analysis (the open-core line,
  the wedge, single-run-free/cross-run-paid) lives in the **separate
  `bot-bottle-console` repo**, not this one — cited here from memory, not
  linked.