Compare commits

..

4 Commits

Author SHA1 Message Date
didericis aa44feea02 docs(research): note on malicious-commit scanning at the git-gate + paid-feature analysis
Adds a research note on whether/how to scan for malicious code (not just
secrets) in commits pushed through the git-gate, and whether the semantic
(LLM) layer is a defensible paid feature.

Verdict: no scanner reliably detects malicious code (undecidable +
adversarial), so the frame is raise-cost + cover-the-obvious + human-gate
the dangerous. Ranked layers: dependency/supply-chain scanning (Socket/OSV/
GuardDog) > heuristic/obfuscation (Semgrep-on-diff) > risk-based human
gating via the existing supervise plane > best-effort LLM diff-review.
Fast scanners inline in the synchronous pre-receive; heavy analysis async.

Monetization: the paid unit is the governed git-egress review bundle
(managed semantic review + web-console human-review flow + RBAC + audit +
cross-run policy), not the raw scanner — which stays OSS like gitleaks.
Extends the egress audit+custody wedge to code artifacts; the supervise
console generalizes across all proposal types (egress, gitleaks, commit
review). Sell the workflow, not the detector's accuracy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 07:00:22 -04:00
didericis f2e2572a40 docs(research): add DX axis — "run Claude yolo-style" — to the sandbox landscape
Adds a "DX: run Claude yolo-style" row to the comparison table plus a note
framing developer experience as a differentiator. The field splits into
wrappers-around-the-agent (bot-bottle, agent-safehouse — one command, the
agent just runs, `--dangerously-skip-permissions` on by default with the
sandbox as the guardrail) vs libraries/services (boxlite, microsandbox,
CubeSandbox, E2B — you wire the agent in via SDK/cluster). agent-safehouse
is the only DX peer, but it's macOS-only Seatbelt with no egress story.
"As easy as native yolo, but actually sandboxed" is the defensible line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:41:26 -04:00
didericis 7069fa225d docs(research): add long-running posture axis to the sandbox landscape
Adds a "Long-running posture" row to the comparison table and an addendum
note contrasting the two models: E2B and CubeSandbox are ephemeral-per-task
(5-min default timeout, tier-capped continuous runtime, duration via
pause/resume + reconnect-by-id), while bot-bottle bottles are persistent,
named, and supervised by default. For agents that run for hours/days this
posture difference matters more than the isolation primitive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:26 -04:00
didericis aa224c4381 docs(research): add CubeSandbox to the sandbox landscape; fix stale bot-bottle self-description
Adds CubeSandbox (Tencent Cloud, Apache 2.0, RustVMM/KVM microVM) to the
agent-sandbox landscape survey: per-project note, comparison-table column,
and a dated addendum on what it means for positioning. CubeSandbox is the
first surveyed project to bundle a connection-level egress allowlist +
audit + in-flight credential custody, but it does NOT do content DLP on
authorized channels — that plus the orchestration layer is where
bot-bottle stays distinctive.

Also corrects two stale self-descriptions the survey (2026-05-11) baked
in and I'd propagated:
- Default isolation is now a VM per bottle (Firecracker microVM on KVM
  Linux, Apple Container on macOS); Docker is only the legacy fallback,
  per _default_backend_name(). Was described as Docker-by-default.
- Outbound DLP is bot-bottle's own mitmproxy egress scanner + gitleaks on
  git push, not pipelock (removed). All references updated; a note
  records the change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:12 -04:00
8 changed files with 428 additions and 352 deletions
+2 -13
View File
@@ -24,19 +24,8 @@ jobs:
- name: Run pylint
run: |
# Pylint's normal exit code is nonzero for any emitted finding,
# regardless of --fail-under. Preserve the full report but enforce
# the aggregate score this workflow promises.
set +e
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' \
| xargs pylint --fail-under=8.0 \
| tee /tmp/pylint-output.txt
set -e
SCORE=$(sed -n \
's/^Your code has been rated at \([-0-9.]*\)\/10.*/\1/p' \
/tmp/pylint-output.txt | tail -1)
test -n "$SCORE"
awk -v score="$SCORE" 'BEGIN { exit !(score >= 8.0) }'
# Run pylint on all Python files in the repo
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
- name: Run pyright
run: |
-33
View File
@@ -1,33 +0,0 @@
name: tracker-policy
on:
issues:
types: [opened, unlabeled]
pull_request:
types: [opened, edited, reopened, synchronized, labeled, unlabeled]
jobs:
label-issue:
if: ${{ github.event_name == 'issues' }}
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/checkout@v4
- name: Ensure the issue has a label
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 scripts/tracker_policy.py label-issue
check-pr:
if: ${{ github.event_name == 'pull_request' }}
runs-on: ubuntu-latest
permissions:
issues: read
pull-requests: read
steps:
- uses: actions/checkout@v4
- name: Require an unlabeled PR linked to an issue
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 scripts/tracker_policy.py check-pr
-9
View File
@@ -171,15 +171,6 @@ When an outbound DLP detector matches a token, the route's `dlp.outbound_on_matc
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Tracker policy
Issues are the canonical work items and own all tracker labels; every issue
must have at least one. Pull requests stay unlabeled and deliberately reference
an issue with `Closes #…`, `Part of #…`, or another form defined in
[`ADR 0005`](docs/decisions/0005-issues-own-tracker-metadata.md). Gitea Actions
enforces the convention for new work from 2026-07-18 onward. Earlier closed
PRs are grandfathered rather than given artificial retrospective issues.
## Trademarks
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
@@ -1,57 +0,0 @@
# ADR 0005: Keep tracker metadata on issues
- **Status:** Accepted
- **Date:** 2026-07-18
- **Deciders:** didericis
## Context
Gitea exposes labels on both issues and pull requests. Applying the same labels
to both copies planning metadata, creates a synchronization obligation, and
makes disagreements between the two records possible. At the same time,
unlabelled objects look accidental unless the repository states which object
owns the metadata.
The repository already uses issues as work items and PRs as implementations of
those work items. At this decision's cutoff, all open PRs reference issues, but
121 of 219 historically merged PRs do not. Manufacturing retrospective issues
for that history would create records that never participated in planning and
would make the issue history less truthful.
## Decision
Issues are the canonical tracker records and own labels. Every issue has at
least one label. An issue opened or left without labels receives
`Status/Needs Triage` automatically until it is classified.
Pull requests carry no labels. Every new PR deliberately references at least
one existing issue in its title or description with one of these forms:
- `Closes #123`, `Fixes #123`, or `Resolves #123` when merging completes it.
- `Part of #123`, `Related to #123`, `Refs #123`, or `References #123` when it
contributes without completing it.
Gitea Actions enforces both PR rules as a status check and repairs the empty
issue-label state. Branch protection makes the PR policy check required.
The policy applies from 2026-07-18 onward. Existing issues may be labelled as
they are encountered, but closed PRs are grandfathered: no retrospective
issues or PR labels are created solely to make history conform.
## Consequences
- Classification, priority, and workflow metadata have one source of truth.
- A PR's issue link is the navigation path to its planning metadata.
- Multi-PR issues do not require copied or synchronized labels.
- `Status/Needs Triage` is an intentional fallback, not a final
classification.
- Direct issue creation remains convenient; automation repairs a missing label
immediately after creation because Gitea has no native required-label rule.
- The required check must be configured in branch protection after this
workflow lands.
## Links
- Issue #405.
- `.gitea/workflows/tracker-policy.yml`.
- `scripts/tracker_policy.py`.
+222 -43
View File
@@ -6,27 +6,49 @@ general AI-agent sandbox / containment projects — some Claude-specific,
some agent-agnostic, some hosted SaaS — and contrasts them with
bot-bottle's design.
Research conducted 2026-05-11.
Research conducted 2026-05-11. CubeSandbox added 2026-07-18 (see its
per-project note and the addendum at the end). Also updated 2026-07-18:
bot-bottle no longer uses **pipelock** — outbound DLP is now bot-bottle's
own (deliberately simple) egress scanner (a mitmproxy addon with custom
detectors, PRD 0017 / 0053), and git-push secret scanning is handled by
**gitleaks** in the git-gate. "pipelock" below has been replaced with the
current mechanism; it survives only in older PRDs as history.
## Summary
Eight projects surveyed. None duplicate bot-bottle's combination of
local Docker, declarative JSON manifest, per-agent egress allowlist via
pipelock, and bottle/agent split. Two clusters stand out:
Nine projects surveyed. None duplicate bot-bottle's combination of
local VM-per-bottle isolation (Firecracker microVM on KVM Linux, Apple
Container on macOS — Docker is now only the legacy fallback), a
declarative JSON manifest, per-agent egress allowlist + outbound-content
DLP via bot-bottle's own egress scanner (plus gitleaks secret-scanning on
git push), and bottle/agent split. Two clusters stand out:
- **Closest neighbours** — agent-safehouse and litterbox: local,
single-user, thin wrappers over an existing OS primitive
(`sandbox-exec`, Podman + Landlock).
- **Different category** — tilde.run (hosted SaaS), boxlite and
microsandbox (microVM libraries for platform builders), endo-familiar
microsandbox (microVM libraries for platform builders), CubeSandbox
(self-hosted multi-tenant microVM service), endo-familiar
(capability-security paradigm, no OS isolation).
The microVM cluster (matchlock, smolmachines, boxlite, microsandbox) is
the most relevant for the v2 isolation discussion in
The microVM cluster (matchlock, smolmachines, boxlite, microsandbox,
CubeSandbox) is the most relevant for the v2 isolation discussion in
[`stronger-isolation-alternatives.md`](stronger-isolation-alternatives.md):
libkrun and Apple's Virtualization.framework have made local microVMs
ergonomic enough that a `"runtime": "microvm"` option on a bottle is now
plausible without a heavy stack.
ergonomic enough that microVMs are **now bot-bottle's default backend**
(Firecracker on KVM Linux, Apple Container on macOS), with Docker kept
only as a legacy fallback for CI / hosts without KVM or Apple Container.
That discussion has since shipped, not just been theorized.
**The one that matters most for positioning is CubeSandbox** — it is the
first surveyed project to ship bot-bottle's would-be wedge (default-deny
egress allowlist + full audit logs + in-flight credential custody so keys
never enter the sandbox) *combined with* per-sandbox microVM isolation,
open-source under Apache 2.0, with Tencent Cloud behind it and 10.4k
stars. It's a self-hosted multi-tenant service for platform builders, not
a single-user declarative tool, so it doesn't collide head-on — but it
narrows the "nobody else bundles egress custody + credential injection"
claim that the monetization positioning leans on. See the addendum.
## Per-project notes
@@ -155,67 +177,105 @@ plausible without a heavy stack.
also supported.
- **Maturity**: Active through April 2026.
### CubeSandbox *(added 2026-07-18)*
- **Source**: https://github.com/TencentCloud/CubeSandbox ;
HN launch https://news.ycombinator.com/item?id=47863430
- **License**: Apache 2.0 (~10.4k stars). By Tencent Cloud; described as
"battle-tested, production-ready" infra already running in Tencent
Cloud. Rust / Go / C.
- **Isolation**: MicroVMs via RustVMM + KVM — "each sandbox gets its own
Guest OS kernel, no Docker shared-kernel escapes." Hardware-level
isolation, dedicated kernel per instance.
- **Locality**: Self-hosted, but **server/cluster-oriented**, not a
single-user local CLI. Deploy guides target PVM cloud VMs, bare metal,
and dev. A single 96-vCPU host is claimed to run 2,000+ concurrent
sandboxes.
- **Agent integration**: **Drop-in E2B SDK replacement** (single env-var
change) — the headline compatibility claim. OpenClaw assistant
integration; general LLM-code execution. Aimed at platform builders,
not one developer's laptop.
- **Config**: Programmatic via the E2B-compatible SDK. No declarative
manifest.
- **Network policy**: This is the striking part — **domain allowlists,
instant block on unauthorized egress, full audit logs, per-sandbox
traffic tokens, policy-routing egress**, enforced by an eBPF-based
virtual switch giving kernel-level network isolation. Closest match yet
to bot-bottle's own default-deny + per-bottle allowlist egress model.
- **Credentials**: **Credential vault** — agents call external APIs / LLMs
while "keys never enter the sandbox, model context, or logs." Same
in-flight-injection idea as matchlock, but productized as a vault.
- **Performance**: <60ms cold start (claimed 2.550× faster than
alternatives), <5MB memory per instance; millisecond snapshot rollback
is upcoming.
- **Maturity**: Open-sourced July 2026 off production Tencent Cloud use;
most-starred project in this set (~10.4k).
## Comparison table
| Axis | bot-bottle | endo-familiar | litterbox | agent-safehouse | matchlock | tilde.run | boxlite | microsandbox | smolmachines |
|---|---|---|---|---|---|---|---|---|---|
| Isolation | Docker + internal net + pipelock; gVisor if present | Object-capability (no OS isolation) | Podman + opt. Landlock | macOS `sandbox-exec` | MicroVM (Firecracker / Virt.fw) | Hosted container (unverified) | MicroVM (KVM / Hypervisor.fw) | MicroVM (libkrun) | MicroVM (libkrun / KVM) |
| Local vs hosted | Local | Local | Local (Linux) | Local (macOS) | Local | Hosted SaaS | Local | Local | Local |
| Open source | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | MIT | No | Apache 2.0 | Apache 2.0 | Apache 2.0 |
| Agent target | Claude Code | Generic (demo) | Generic | Multi-agent wrapper | Generic (+ Claude/OpenAI SDKs) | Claude focus | Generic | Claude + Cursor (MCP/Skills) | Generic (AGENTS.md) |
| Network policy | Default-deny via pipelock + per-bottle allowlist + DLP | Capability model only | Limited | Not addressed | Default-deny + allowlist + secret-injecting proxy | Default-deny + logging | Per-VM net (unverified) | Not documented | Off by default + allowlist |
| Parallel agents | Yes (one bottle per agent) | n/a | Not addressed | One at a time | Multiple VMs | Yes (dashboard) | SDK-level | SDK-level | Architectural |
| Config | JSON manifest (bottles + agents) | Programmatic refs | CLI wizard | Profile files / shell fns | CLI / SDK | DSL + CLI + SDK | SDK | CLI / SDK / MCP | TOML Smolfile |
| Maturity | Active May 2026 | Research (2022+) | Early (~66 ⭐) | Active (~1.4k ⭐) | Experimental (~574 ⭐) | Private preview | YC, ~4.7k ⭐ | YC, ~6k ⭐, beta | ~3.1k ⭐ |
| Axis | bot-bottle | endo-familiar | litterbox | agent-safehouse | matchlock | tilde.run | boxlite | microsandbox | smolmachines | CubeSandbox |
|---|---|---|---|---|---|---|---|---|---|---|
| Isolation | MicroVM per bottle default (Firecracker/KVM on Linux, Apple Container on macOS) + own egress DLP scanner; Docker legacy fallback, gVisor there if present | Object-capability (no OS isolation) | Podman + opt. Landlock | macOS `sandbox-exec` | MicroVM (Firecracker / Virt.fw) | Hosted container (unverified) | MicroVM (KVM / Hypervisor.fw) | MicroVM (libkrun) | MicroVM (libkrun / KVM) | MicroVM (RustVMM / KVM) |
| Local vs hosted | Local | Local | Local (Linux) | Local (macOS) | Local | Hosted SaaS | Local | Local | Local | Self-hosted (server/cluster) |
| Open source | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | MIT | No | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 |
| Agent target | Claude Code | Generic (demo) | Generic | Multi-agent wrapper | Generic (+ Claude/OpenAI SDKs) | Claude focus | Generic | Claude + Cursor (MCP/Skills) | Generic (AGENTS.md) | E2B-compatible (platform builders) |
| Network policy | Default-deny via own egress scanner + per-bottle allowlist + content DLP + gitleaks on git push | Capability model only | Limited | Not addressed | Default-deny + allowlist + secret-injecting proxy | Default-deny + logging | Per-VM net (unverified) | Not documented | Off by default + allowlist | Default-deny allowlist + instant egress block + audit logs + per-sandbox tokens (eBPF) + credential vault |
| Parallel agents | Yes (one bottle per agent) | n/a | Not addressed | One at a time | Multiple VMs | Yes (dashboard) | SDK-level | SDK-level | Architectural | Yes (2,000+/host claimed) |
| Long-running posture | Persistent by default (named, supervised) | n/a (demo) | Session (up while in use) | Per-invocation | Ephemeral VM per run | Per-run (versioned) | Ephemeral + snapshot/fork | Ephemeral / on-demand | Named persistent by default | Ephemeral + auto pause/resume |
| DX: run Claude yolo-style | One command → interactive yolo Claude (`start <agent>`, `--dangerously-skip-permissions` default) | n/a (lib demo) | Wizard + build, then run claude inside (Linux only) | One-command wrapper (`safehouse claude --dangerously-skip-permissions`) | CLI: run a cmd in a VM (not a Claude wrapper) | Hosted (`tilde exec`), not local-native | SDK code required (build the run yourself) | CLI/MCP: sandbox-as-a-tool for the agent, not a wrapper around it | SSH into a named machine, run claude there | Stand up a cluster + drive via E2B SDK |
| Config | JSON manifest (bottles + agents) | Programmatic refs | CLI wizard | Profile files / shell fns | CLI / SDK | DSL + CLI + SDK | SDK | CLI / SDK / MCP | TOML Smolfile | E2B-compatible SDK |
| Maturity | Active May 2026 | Research (2022+) | Early (~66 ⭐) | Active (~1.4k ⭐) | Experimental (~574 ⭐) | Private preview | YC, ~4.7k ⭐ | YC, ~6k ⭐, beta | ~3.1k ⭐ | Tencent, prod, ~10.4k ⭐ |
## What's closest, what's different
**Closest in design and scope.** agent-safehouse and litterbox sit
nearest bot-bottle: local, single-user, thin wrappers over an
existing OS primitive, low-dep. The split is the isolation primitive —
bot-bottle uses Docker + pipelock egress (plus gVisor where
available); agent-safehouse uses `sandbox-exec`; litterbox uses Podman +
Landlock. matchlock and smolmachines are spiritually close on the
*policy* side (default-deny net, per-host allowlist) but use microVMs
instead of containers.
bot-bottle now defaults to a VM per bottle (Firecracker microVM on KVM
Linux, Apple Container on macOS) with its own DLP-scanning egress proxy,
keeping Docker only as a legacy fallback; agent-safehouse uses
`sandbox-exec`; litterbox
uses Podman + Landlock. matchlock and smolmachines are close on *both* the
policy side (default-deny net, per-host allowlist) and — now that
bot-bottle has moved off containers-by-default — the microVM isolation
primitive.
**Solving a different problem.** tilde.run is hosted SaaS for team /
production agent pipelines with data-versioned rollback — explicitly
opposite to bot-bottle's "infrastructure I control" goal. boxlite and
microsandbox are infrastructure libraries aimed at platform builders
embedding sandboxes into agent frameworks; they would be a *backend*
bot-bottle could call, not a competitor to its manifest layer.
endo-familiar is in a different paradigm entirely: capability passing
rather than kernel boundaries.
opposite to bot-bottle's "infrastructure I control" goal. boxlite,
microsandbox, and CubeSandbox are infrastructure libraries/services aimed
at platform builders embedding sandboxes into agent frameworks; they
would be a *backend* bot-bottle could call, not a competitor to its
manifest layer. endo-familiar is in a different paradigm entirely:
capability passing rather than kernel boundaries.
## Borrowable ideas
What bot-bottle already has that the survey suggested as
differentiators:
- Default-deny egress with a per-agent allowlist (pipelock).
- Default-deny egress with a per-agent allowlist (own egress scanner).
- DLP scanning of outbound traffic.
- Bottle / agent split (manifest layer above the isolation primitive).
- gVisor auto-detection on Linux.
Ideas worth considering, without abandoning the Python-stdlib-first / local-Docker
stance:
Ideas worth considering, without abandoning the Python-stdlib-first /
local, single-operator stance:
1. **Per-use SSH key confirmation** (from litterbox). Even with
KnownHostKey pinning and pipelock egress, a wrapper SSH agent that
KnownHostKey pinning and the egress DLP scanner, a wrapper SSH agent that
prompts on each key use (e.g. via `osascript` / `notify-send`) would
catch an agent doing something off-policy with a key it legitimately
holds. Pure-stdlib, no new deps.
2. **In-flight secret injection** (from matchlock). Pipelock already
does egress allowlisting and DLP; teaching it to *inject* tokens at
2. **In-flight secret injection** (from matchlock). The egress scanner
already does allowlisting and DLP; teaching it to *inject* tokens at
proxy time so e.g. `GITEA_TOKEN` never appears in the container's
env would close the "agent reads its own env and exfiltrates" path.
Fits the existing pipelock architecture.
3. **MicroVM backend as an opt-in bottle type** — already on the radar
in `stronger-isolation-alternatives.md`. microsandbox, smolmachines,
and matchlock all show that libkrun + Apple's
Virtualization.framework is ergonomic enough that a
`"runtime": "microvm"` field on a bottle is plausible without a heavy
stack.
Fits the existing egress-proxy architecture.
3. **MicroVM backend**~~on the radar~~ **shipped since this survey.**
microVMs are now bot-bottle's default (Firecracker on KVM Linux, Apple
Container on macOS); Docker is the legacy fallback. The libkrun / Apple
Virtualization.framework ergonomics that microsandbox, smolmachines,
and matchlock demonstrated turned out to be enough to make it the
default rather than an opt-in.
Not worth borrowing: the SDK-first programmatic API style of boxlite /
microsandbox (cuts against the declarative-manifest stance), and the
@@ -230,3 +290,122 @@ hosted-SaaS dashboard model of tilde.run (cuts against the
- The `superradcompany/microsandbox` URL in the original prompt
redirects to `microsandbox/microsandbox`; the surveyed project is the
same.
- CubeSandbox performance/scale numbers (<60ms cold start, <5MB/instance,
2,000+ sandboxes per 96-vCPU host) are the project's own launch claims,
not independently verified here.
## Addendum 2026-07-18 — CubeSandbox and the positioning read
CubeSandbox (Tencent Cloud, Apache 2.0, ~10.4k stars, HN launch
[#47863430](https://news.ycombinator.com/item?id=47863430)) is the first
project in this survey to combine, in one open-source stack, everything
bot-bottle treated as its differentiator:
- **Egress custody (connection level)** — default-deny domain allowlist
(L7 domain/SNI filtering), instant block on unauthorized egress,
per-sandbox traffic tokens, full audit logs of destinations (eBPF
virtual switch, "CubeVS"). This matches bot-bottle's egress scanner at
the *connection level*, productized — see the one thing it does **not**
match, below.
- **Credential custody** — a vault where keys "never enter the sandbox,
model context, or logs." This is the in-flight-injection idea from
matchlock, but as a first-class feature, and it's exactly the
cross-vendor "egress audit + custody" wedge the monetization
positioning treats as the one defensible moat.
- **Isolation on par with bot-bottle's current default** — a dedicated
guest kernel per sandbox (RustVMM/KVM). bot-bottle now defaults to the
same class of boundary (Firecracker microVM / Apple Container), so this
is parity, not an edge; CubeSandbox's remaining edge is running that
per-kernel isolation multi-tenant at scale on one host.
The one axis CubeSandbox does **not** cover — and where bot-bottle stays
distinctive:
- **Content DLP on *authorized* channels.** CubeSandbox's egress control
is connection-level: it decides *whether* a destination is allowed and
logs it, and its vault keeps *injected* credentials out of the sandbox
entirely. Neither inspects the *payload* of traffic to an allowed
destination. So an agent that exfiltrates over a permitted channel —
pasting a repo's contents, an agent-derived secret, or PHI into an
allowed API/domain — is not caught by CubeSandbox. bot-bottle's own
egress DLP scanner does scan that: response + websocket content against
the resolved per-flow config, with per-bottle token redaction (see
recent egress commits). The vault
approach is arguably *stronger* for the specific case of pre-known
injected credentials (they can't leak if they were never present), but
it is not a substitute for content inspection of everything else.
**Long-running posture — a sharper axis than raw isolation.** E2B and
CubeSandbox are *ephemeral-per-task* by design; a long-running agent is an
architected pattern on top, not the default. E2B: 5-minute default
timeout, continuous runtime tier-capped (~1h Hobby / ~24h Pro), duration
achieved via **pause/resume** (preserves filesystem + memory + processes;
reconnect by sandbox ID via `Sandbox.connect()`; resume resets the timeout
to 5 min; auto-pause via `on_timeout: "pause"`). CubeSandbox mirrors this
(E2B drop-in) with first-class auto pause/resume and hundred-ms
checkpoint/fork — and, self-hosted, sets its own timeout policy with no
vendor tier caps. bot-bottle inverts the model: a bottle is **persistent,
named, and supervised by default** — long-running *is* the default, not a
session-management loop over pause/resume. smolmachines is the other
persistent-by-default project in this set. For anyone building agents that
run for hours/days, this posture difference matters more than the
isolation primitive.
**DX — the "run Claude yolo-style" bar.** The reason `claude
--dangerously-skip-permissions` is so widely used is DX: it's one command
and the agent just goes. The bottle thesis is to make a *sandboxed* run
that easy — `start <agent>` builds the image on first run and drops you
into an interactive Claude session that already has
`--dangerously-skip-permissions` on by default
(`contrib/claude/agent_provider.py`), with the sandbox as the guardrail
instead of per-action prompts. On this axis the field splits cleanly:
- **Wrappers around the agent** (as-easy-as-native): bot-bottle and
**agent-safehouse** (`safehouse claude --dangerously-skip-permissions`).
These *are* the run-Claude experience. agent-safehouse is the real DX
peer — but it's macOS-only Seatbelt, single-run, and doesn't address
network egress; bot-bottle adds VM-grade isolation, egress DLP, and
persistent/parallel bottles across macOS + Linux.
- **Libraries / services** (you build the run yourself): boxlite,
microsandbox, CubeSandbox, E2B. These hand you an SDK or a cluster and
expect you to wire the agent in — powerful for platform builders,
heavyweight for "just run Claude on my laptop." microsandbox's MCP/Skills
angle is *sandbox-as-a-tool the agent calls*, which is the inverse of
wrapping the agent.
- **In between:** litterbox (wizard + build, Linux only), smolmachines
(SSH into a named machine), matchlock (run a command in a VM).
So DX is a genuine bot-bottle differentiator, and the only project that
matches it (agent-safehouse) does so with materially weaker isolation and
no egress story. "As easy as native yolo, but actually sandboxed" is a
defensible one-liner.
Why it still doesn't collide head-on:
1. **Shape.** CubeSandbox is a *multi-tenant service for platform
builders* (drop-in E2B replacement, SDK-driven, 2,000 sandboxes on a
box). bot-bottle is a *single-operator, declarative-manifest tool for
the infrastructure I run*. Different buyer, different ergonomics — no
JSON manifest, no bottle/agent split, no "one command on my laptop."
2. **Backend, not competitor.** Like boxlite/microsandbox, CubeSandbox is
something bot-bottle could sit *on top of* — a `"runtime": "microvm"`
or `"runtime": "cubesandbox"` backend under the manifest layer — while
keeping the manifest, the bottle/agent split, and the local,
single-operator default.
Why it matters anyway:
- The "nobody else bundles connection-level egress allowlist + audit +
in-flight credential custody" line is **no longer true for the
primitive** — a well-funded, 10k-star open-source project now ships it.
But **content DLP on authorized channels is still not matched** (see
above), and neither is the *layer above* the primitive (declarative
manifest, cross-vendor orchestration, operator UX, the
phone-control/dashboard north star). Those two — outbound-payload DLP
and the orchestration layer — are where the defensible ground now sits;
the connection-level allowlist + vault mechanism, on its own, is no
longer differentiating. Revisit the monetization open/paid line with
that in mind.
- Worth a closer look at **how** CubeSandbox does credential injection
and per-sandbox egress tokens (eBPF virtual switch vs. bot-bottle's
mitmproxy egress proxy) before the next iteration of bot-bottle's
in-flight-secret feature — see borrowable idea #2 above.
+204
View File
@@ -0,0 +1,204 @@
# Scanning for malicious commits at the git-gate — and whether the semantic layer is a paid feature
A sandboxed agent's most durable way out isn't live exfiltration — it's the
**artifact it pushes**. Code committed through the git-gate leaves the
sandbox and runs *later*, in CI or production, with none of the bottle's
isolation or egress control. gitleaks covers *secrets* on that channel
(see [`git-gate` pre-receive](../../bot_bottle/git_gate_render.py)); this
note asks the harder question — can we scan for *malicious code* in
commits — and the product question the answer raises: is the semantic
(LLM) layer a decent paid feature on "git egress," alongside a paid
human-review console for the supervise flow?
## Summary
- **No scanner reliably detects malicious code.** It's undecidable in
general and adversarial in practice — a capable agent hides a backdoor
from any pattern scanner (and from human reviewers). So the frame is not
"detect malice"; it's **raise the cost, cover the common/obvious cases,
and put a human on the high-leverage ones.**
- **Layered, ranked by ROI:** (1) supply-chain / malicious-dependency
scanning (Socket, OSV-Scanner, GuardDog) — highest ROI, best git fit;
(2) fast heuristic / obfuscation / dangerous-sink scanning
(Semgrep-on-diff) — gitleaks-shaped; (3) **risk-based human gating** via
the existing supervise plane on high-risk file classes; (4) **LLM
semantic diff-review** as a best-effort backstop for novel backdoors.
- **Architecture:** the git-gate's `pre-receive` is *synchronous* (it
blocks the push), so run only fast scanners inline and push heavy
analysis (CodeQL, full LLM review) to an **async post-push CI lane** that
can't hold the developer hostage.
- **Monetization verdict: yes — but the paid unit is the *governed
git-egress review bundle*, not the raw scanner.** The pre-receive
mechanics and the fast OSS scanners stay free (open-core social
contract, same as gitleaks). The sell is the control-plane workflow
around them: **managed semantic review + a web-console human-review flow
+ RBAC + audit retention + cross-run policy.** A bare per-push scan is
borderline "single-run = free" by our own heuristic, and its detection
accuracy is not something to sell on. Sell the *workflow and the
governance*, not the detector.
## The threat: temporal escape
This is the one containment fundamentally can't close. bot-bottle's VM
isolation + egress DLP shrink *live* exfiltration, but the git push
channel launders a payload past all of it: the agent commits code that
looks benign at review time and does its work later, outside the
perimeter — the "blind-XSS"-style temporal escape that both the "3 reasons
sandboxing won't secure your AI agent" talk and Embroidery's "In sandboxes
we (shouldn't) trust" call the sharpest gap. gitleaks gates *secrets* on
this channel; malicious *code* is the uncovered surface, and it's the
behavioral half of the isolation-vs-behavioral split noted in
[`agent-sandbox-landscape.md`](agent-sandbox-landscape.md).
## Why it's fundamentally hard
Distinguishing malicious from legitimate code is undecidable in general
and adversarial in the specific. The underhanded-code contests show humans
hide backdoors from *human reviewers*; a scanner reduces to regex/dataflow
heuristics a motivated agent writes around. So "catch malicious commits"
is the wrong success metric. The right ones: *how much did we raise the
cost*, *did we cover the common and the obvious*, and *did a human see the
dangerous changes before they left*.
## Landscape of git-pluggable approaches (ranked by ROI)
1. **Supply-chain / malicious-dependency scanning — highest ROI.** The
most likely agent-introduced threat is a malicious or typosquatted
*dependency*, not a hand-written backdoor — and it's far more
detectable. **Socket** (behavioral analysis of packages: install
scripts, obfuscation, unexpected network/fs access, typosquats) is the
closest thing to malicious-intent detection that actually works;
**OSV-Scanner** (known-bad DB, fast) and **GuardDog** (PyPI/npm
heuristics) complement it. Trigger only when a commit changes a
lockfile/manifest → block or escalate.
2. **Fast heuristic / obfuscation / dangerous-sink scanning.** **Semgrep**
with a curated ruleset, scanning the *diff* (fast enough for
pre-receive). Catches obfuscation (`eval(atob(...))`, base64/hex blobs,
minified source), exfil primitives (`curl | sh`, reverse shells,
DNS-exfil), and dangerous sinks (`shell=True`, `pickle.loads`, unsafe
`yaml.load`, `child_process.exec`). Tune noisy rules to *escalate*, not
block.
3. **Risk-based human gating — the most robust control, already
half-built.** The git-gate already escalates `# gitleaks:allow`
findings to a supervisor proposal and blocks the push pending approval
(`supervise_gitleaks_allow`, [`git_gate_render.py`](../../bot_bottle/git_gate_render.py)).
Extend the same flow to **high-risk file classes**: any commit touching
CI/build/deploy scripts, auth/crypto code, egress config, or
adding/changing dependencies → route to `./cli.py supervise`. This is
attribution/policy, not detection, and it's the strongest thing here —
a human on exactly the temporal-escape surfaces.
4. **LLM semantic diff-review — the behavioral backstop.** The only
approach with a shot at a novel, benign-looking backdoor: a model
reviews the diff for a backdoor / exfiltration / obfuscated payload /
dangerous behavior. This is the behavioral-detection thesis applied to
the *artifact* (the diff) rather than the agent's reasoning. Real
caveats: false positives and negatives, latency/cost, it can be
prompt-injected by hostile content *in the diff itself*, and it invites
over-trust. Useful, never a guarantee.
**Not for this job:** CodeQL, Trivy, Grype, Bandit. They find *known
vulns and insecure patterns* (bugs), not deliberate backdoors, and the
powerful ones (CodeQL taint) need a build + database — too heavy for a
synchronous gate. They belong in the async CI lane if at all.
## Fit into bot-bottle's git-gate
The `pre-receive` hook today is: gitleaks-scan each ref → escalate
`# gitleaks:allow` findings to supervise → forward to upstream
([`git_gate_render.py`](../../bot_bottle/git_gate_render.py)). The
additions slot in cleanly:
- **Inline (fast), before forward:** a dep-scan phase (on manifest/lockfile
change) and a Semgrep-diff phase. Findings block or open a supervise
proposal, same shape as gitleaks.
- **New supervise tool types** alongside the existing
`egress-block/allow`, `gitleaks-allow`, `egress-token-allow`
([`supervise_types.py`](../../bot_bottle/supervise_types.py)) — e.g. a
`commit-review` proposal for risky-file-class gating and for semantic
review. The supervise plane is already the right abstraction; this is
another *producer* feeding it, and [`supervise_server.py`](../../bot_bottle/supervise_server.py)
(JSON-RPC) is already the console backend.
- **Async lane (heavy):** full LLM review + any CodeQL run out of band
after the push, feeding the same review/audit surface, so the
synchronous gate stays fast.
## The product question: paid feature on git egress?
Restating the open-core line bot-bottle runs on: *give away the
sandbox/runtime, charge for the control plane; single-run/single-node =
free, cross-run aggregation + central enforcement + identity/fleet = paid;
the moat is uniform egress audit + secret custody + policy across
untrusted agents.*
Against that line, the split is clean:
**Free (OSS runtime — the trust funnel):**
- the `pre-receive` gate mechanics and gitleaks;
- wiring the OSS scanners (Socket CLI / OSV-Scanner / Semgrep);
- the CLI supervise flow.
Keeping the raw scanners free is the same social contract as gitleaks and
preserves the bottom-up distribution funnel.
**Paid (the governed git-egress bundle — the control plane):**
- **Managed semantic diff-review** — hosted inference + a curated,
maintained malicious-pattern/policy set. This is *capability* (metered),
not *insurance* — the thing individuals actually pay for. Position it as
**governed code-egress review**, not "we resell inference" (the
monetization notes explicitly warn against reselling compute).
- **The web-console supervise/review flow — the strongest anchor.** Turn
the CLI `./cli.py supervise` approval into a real review surface:
rendered diff + finding context, approve/reject, **who-approved audit
trail, RBAC on approvers, mobile/phone-control** (ties to the
dashboard/vault north star). This is "central enforcement +
identity/fleet = paid" almost verbatim — and it generalizes across
*every* supervise proposal (egress block/allow, gitleaks-allow,
commit-review), so it's worth building for the whole plane, with the
semantic check as one producer.
- **Cross-run governance:** fleet-wide policy for what escalates,
review-decision history/search/export, and drift alerts.
**Why it fits the moat rather than bolting on:** a git push *is* an egress
channel. A semantic review + human approval + audit on it extends the
uniform "egress audit + custody + policy across untrusted agents" wedge to
**code artifacts** — the same product, applied to the one channel gitleaks
only half-covers. That's on-moat, not a detour.
**The honest nuance (don't oversell):** a bare per-push LLM scan is
arguably *free* by the single-run heuristic, and its detection accuracy is
not defensible to charge for. The paid value is the **governance around
it** — the console, RBAC, audit retention, cross-run policy — plus the
managed capability. Sell the *review-and-approve-and-audit workflow*; let
the detector be explicitly best-effort. And per the monetization
guardrail, the "anti-corporate" free crowd must not veto these team
features: the review console + RBAC + audit *are* the monetization.
## Recommendation
1. **Land the free layer first.** Add the dep-scan and Semgrep-diff phases
to `pre-receive`, and extend supervise to risky-file-class gating —
reuses existing machinery, immediate value, stays OSS.
2. **Build the supervise web console** over `supervise_server`'s JSON-RPC
(already the Phase-1 move in the monetization path). This is the paid
anchor and it serves *all* proposal types, not just commit review.
3. **Add managed semantic diff-review as a paid producer** feeding that
console — "governed code-egress review," metered, explicitly
best-effort on detection.
4. **Don't oversell detection.** Market the workflow (review + approve +
audit) and the cross-run policy/RBAC, where the value is real and
defensible; keep the raw scanners open.
## Sources / references
- [`agent-sandbox-landscape.md`](agent-sandbox-landscape.md) — the
egress-DLP gap and isolation-vs-behavioral framing.
- Git-gate internals: [`git_gate_render.py`](../../bot_bottle/git_gate_render.py),
[`supervise_types.py`](../../bot_bottle/supervise_types.py),
[`supervise_server.py`](../../bot_bottle/supervise_server.py).
- External tools: Socket (socket.dev), OSV-Scanner (google/osv-scanner),
GuardDog (DataDog/guarddog), Semgrep (semgrep/semgrep).
- Threat framing: "3 reasons sandboxing won't secure your AI agent"
(youtube TsYDazwHJ6U); Embroidery, "In sandboxes we (shouldn't) trust."
- The authoritative monetization/positioning analysis (the open-core line,
the wedge, single-run-free/cross-run-paid) lives in the **separate
`bot-bottle-console` repo**, not this one — cited here from memory, not
linked.
-135
View File
@@ -1,135 +0,0 @@
#!/usr/bin/env python3
"""Enforce the repository's issue/PR metadata policy in Gitea Actions."""
from __future__ import annotations
import argparse
import json
import os
import re
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any
ISSUE_REFERENCE = re.compile(
r"(?im)\b(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?|part\s+of|"
r"related\s+to|refs?|references)\s+#(\d+)\b"
)
TRIAGE_LABEL = "Status/Needs Triage"
def deliberate_issue_numbers(title: str, body: str) -> set[int]:
"""Return same-repository issue numbers referenced intentionally."""
return {int(match) for match in ISSUE_REFERENCE.findall(f"{title}\n{body}")}
class GiteaApi:
"""Small API client using the Actions-provided repository token."""
def __init__(self, api_url: str, repository: str, token: str) -> None:
self.base = f"{api_url.rstrip('/')}/repos/{repository}"
self.token = token
def request(self, method: str, path: str, payload: object | None = None) -> Any:
data = None if payload is None else json.dumps(payload).encode()
request = urllib.request.Request(
f"{self.base}{path}",
data=data,
method=method,
headers={
"Authorization": f"token {self.token}",
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(request, timeout=15) as response:
if response.status == 204:
return None
return json.load(response)
def check_pull_request(event: dict[str, Any], api: GiteaApi) -> list[str]:
"""Return policy violations for a pull_request event."""
pull = event["pull_request"]
errors: list[str] = []
labels = pull.get("labels") or []
if labels:
errors.append(
"PRs must be unlabeled; put tracker metadata on the linked issue "
f"(found: {', '.join(label['name'] for label in labels)})."
)
numbers = deliberate_issue_numbers(pull.get("title", ""), pull.get("body", ""))
if not numbers:
errors.append(
"PR must reference an issue with Closes/Fixes/Resolves #N, "
"Part of #N, Related to #N, Refs #N, or References #N."
)
return errors
real_issues = 0
for number in sorted(numbers):
try:
item = api.request("GET", f"/issues/{number}")
except urllib.error.HTTPError as error:
if error.code == 404:
errors.append(f"Referenced issue #{number} does not exist.")
continue
raise
if item.get("pull_request") is not None:
errors.append(f"#{number} is a pull request, not an issue.")
else:
real_issues += 1
if not real_issues and not errors:
errors.append("PR must reference at least one real issue.")
return errors
def ensure_issue_label(event: dict[str, Any], api: GiteaApi) -> bool:
"""Apply the triage label if an issue event leaves the issue unlabeled."""
issue = event["issue"]
if issue.get("pull_request") is not None or issue.get("labels"):
return False
labels = api.request("GET", "/labels?limit=100")
triage = next((label for label in labels if label["name"] == TRIAGE_LABEL), None)
if triage is None:
raise RuntimeError(f"repository label {TRIAGE_LABEL!r} does not exist")
api.request("POST", f"/issues/{issue['number']}/labels", {"labels": [triage["id"]]})
return True
def _load_event(path: str) -> dict[str, Any]:
return json.loads(Path(path).read_text(encoding="utf-8"))
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("command", choices=("check-pr", "label-issue"))
parser.add_argument("--event", default=os.environ.get("GITHUB_EVENT_PATH"))
args = parser.parse_args()
if not args.event:
parser.error("--event or GITHUB_EVENT_PATH is required")
api = GiteaApi(
os.environ["GITHUB_API_URL"],
os.environ["GITHUB_REPOSITORY"],
os.environ["GITHUB_TOKEN"],
)
event = _load_event(args.event)
if args.command == "check-pr":
errors = check_pull_request(event, api)
if errors:
print("\n".join(f"::error::{error}" for error in errors))
return 1
print("PR tracker policy passed.")
return 0
changed = ensure_issue_label(event, api)
print(f"Applied {TRIAGE_LABEL}." if changed else "Issue already has a label.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
-62
View File
@@ -1,62 +0,0 @@
import unittest
from unittest.mock import Mock
from scripts.tracker_policy import (
TRIAGE_LABEL,
check_pull_request,
deliberate_issue_numbers,
ensure_issue_label,
)
class TestDeliberateIssueNumbers(unittest.TestCase):
def test_accepts_completing_and_noncompleting_forms(self):
self.assertEqual(
deliberate_issue_numbers("Fixes #12", "Part of #14; refs #15"),
{12, 14, 15},
)
def test_does_not_treat_incidental_number_as_link(self):
self.assertEqual(deliberate_issue_numbers("Audit #12", "See PR #14"), set())
class TestCheckPullRequest(unittest.TestCase):
def test_accepts_unlabelled_pr_linked_to_real_issue(self):
api = Mock()
api.request.return_value = {"number": 12, "pull_request": None}
event = {"pull_request": {"title": "Change", "body": "Part of #12", "labels": []}}
self.assertEqual(check_pull_request(event, api), [])
def test_rejects_labels_and_pr_reference(self):
api = Mock()
api.request.return_value = {"number": 12, "pull_request": {}}
event = {
"pull_request": {
"title": "Change",
"body": "Closes #12",
"labels": [{"name": "Kind/Bug"}],
}
}
errors = check_pull_request(event, api)
self.assertEqual(len(errors), 2)
self.assertIn("unlabeled", errors[0])
self.assertIn("not an issue", errors[1])
class TestEnsureIssueLabel(unittest.TestCase):
def test_adds_triage_label_to_unlabelled_issue(self):
api = Mock()
api.request.side_effect = [[{"id": 55, "name": TRIAGE_LABEL}], None]
event = {"issue": {"number": 405, "labels": [], "pull_request": None}}
self.assertTrue(ensure_issue_label(event, api))
api.request.assert_any_call("POST", "/issues/405/labels", {"labels": [55]})
def test_leaves_labelled_issue_unchanged(self):
api = Mock()
event = {"issue": {"number": 405, "labels": [{"name": "Kind/Documentation"}]}}
self.assertFalse(ensure_issue_label(event, api))
api.request.assert_not_called()
if __name__ == "__main__":
unittest.main()