From 4302678f3ea4eb074de8ca4eb5042f361a15c177 Mon Sep 17 00:00:00 2001 From: claude Date: Sat, 18 Jul 2026 19:11:14 +0000 Subject: [PATCH 1/3] docs(research): expand sandbox landscape with 6 new tools; add agent-tailored policy axis MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Isolation tools added: Cleanroom (Buildkite), container-use (Dagger), Docker sbx, Anthropic srt. Governance/pre-action layers added as a separate section: Microsoft Agent Governance Toolkit (per-agent DID + YAML policy + trust score), Open Agent Passport (declarative policy + cryptographic audit). Comparison table: 14 → 14 columns; new Agent-tailored policy row added. Second addendum covers competitive position on role-tailoring, Docker sbx as new DX-class competitor, and borrowable ideas (trust-score decay, live network TUI, cryptographic audit chain). Discourse note: adds Per-agent role tailoring to "What it covers well" with competitive comparison table across 9 tools. --- docs/research/agent-sandbox-landscape.md | 283 ++++++++++++++++-- .../hn-agent-safety-discourse-july-2026.md | 37 +++ 2 files changed, 295 insertions(+), 25 deletions(-) diff --git a/docs/research/agent-sandbox-landscape.md b/docs/research/agent-sandbox-landscape.md index 23665f3..193cbfb 100644 --- a/docs/research/agent-sandbox-landscape.md +++ b/docs/research/agent-sandbox-landscape.md @@ -14,22 +14,34 @@ detectors, PRD 0017 / 0053), and git-push secret scanning is handled by **gitleaks** in the git-gate. "pipelock" below has been replaced with the current mechanism; it survives only in older PRDs as history. +Updated again 2026-07-18: six additional tools added (Cleanroom, +container-use, Docker sbx, Anthropic srt, Microsoft AGT, Open Agent +Passport); an **Agent-tailored policy** row added to the comparison table; +a separate Governance layers section added for AGT and OAP. See the +second addendum at the end. + ## Summary -Nine projects surveyed. None duplicate bot-bottle's combination of -local VM-per-bottle isolation (Firecracker microVM on KVM Linux, Apple -Container on macOS — Docker is now only the legacy fallback), a -declarative JSON manifest, per-agent egress allowlist + outbound-content -DLP via bot-bottle's own egress scanner (plus gitleaks secret-scanning on -git push), and bottle/agent split. Two clusters stand out: +Fifteen projects surveyed across two categories: isolation/sandbox tools +and governance/pre-action authorization layers (the latter don't provide +VM or container isolation but do per-agent policy enforcement at the +tool-call level). None duplicate bot-bottle's combination of local +VM-per-bottle isolation, a declarative per-role manifest, per-agent +egress allowlist + outbound-content DLP, bottle/agent split, and the +composable `extends:` policy model. Three clusters stand out: - **Closest neighbours** — agent-safehouse and litterbox: local, single-user, thin wrappers over an existing OS primitive (`sandbox-exec`, Podman + Landlock). -- **Different category** — tilde.run (hosted SaaS), boxlite and - microsandbox (microVM libraries for platform builders), CubeSandbox +- **Different category (isolation)** — tilde.run (hosted SaaS), boxlite + and microsandbox (microVM libraries for platform builders), CubeSandbox (self-hosted multi-tenant microVM service), endo-familiar (capability-security paradigm, no OS isolation). +- **New: governance/pre-action layers** — Microsoft AGT and Open Agent + Passport (OAP): framework-embedded tool-call interceptors with + per-agent declarative policy. Closest competitors on agent-tailored + policy, but operate at the tool-call level rather than providing + network/filesystem isolation; they complement rather than substitute. The microVM cluster (matchlock, smolmachines, boxlite, microsandbox, CubeSandbox) is the most relevant for the v2 isolation discussion in @@ -210,20 +222,157 @@ claim that the monetization positioning leans on. See the addendum. - **Maturity**: Open-sourced July 2026 off production Tencent Cloud use; most-starred project in this set (~10.4k). +### Cleanroom *(added 2026-07-18)* +- **Source**: https://github.com/buildkite/cleanroom +- **License**: Apache 2.0 +- **Isolation**: MicroVM — Firecracker on Linux, Virtualization.framework + on macOS. Digest-pinned OCI images. +- **Locality**: Self-hosted server (CI-oriented). +- **Agent integration**: Generic process sandbox; CI-first, not a + Claude/agent wrapper. +- **Config**: `cleanroom.yaml` in the repo being sandboxed defines egress + rules, resources, and network policy. Cleanroom resolves this from the + commit being run. +- **Network policy**: Default-deny + per-repo hostname allowlist (resolved + from DNS answers + destination IP:port). Co-hosted services on the same + IP:port are not distinguished. OIDC-backed auth for remote servers. +- **Credentials**: Host-side only; not injected in-flight but not present + in the VM. +- **Notable**: Policy lives in the *repo being sandboxed*, not in an + agent-role definition — closer to per-repo scoping than per-role. + Supports Docker-inside-sandbox (`services.docker.required: true`), OIDC + authorization, suspend/resume lifecycle. +- **Maturity**: Active Buildkite product. + +### container-use *(added 2026-07-18)* +- **Source**: https://github.com/dagger/container-use +- **License**: Apache 2.0 +- **Isolation**: Docker container per agent + git worktree per agent. + Containers share the host kernel; stronger than bare host but weaker + than microVM. +- **Locality**: Local. +- **Agent integration**: MCP stdio server — Claude Code, Cursor, Windsurf. + `claude mcp add container-use -- container-use stdio`. +- **Config**: None for security policy. Environments are provisioned on + demand; no allowlist or credential config. +- **Network policy**: Not addressed. +- **Notable**: Per-agent git branches (`container-use/`); + parallel agents without filesystem conflict; real-time log visibility + and terminal attach for intervention; git-based review workflow. + Oriented toward parallel development safety, not security containment. +- **Maturity**: Early development, active. + +### Docker sbx *(added 2026-07-18)* +- **Source**: Docker proprietary (`sbx` CLI, separate from `docker`). +- **License**: Proprietary. +- **Isolation**: MicroVM (Docker's own implementation) — each session gets + its own kernel, Docker daemon inside the VM, and filesystem. +- **Locality**: Local (macOS and Windows; does not require Docker Desktop). +- **Agent integration**: Explicit wrapper — Claude Code, Codex, Gemini + CLI, Copilot CLI, Kiro. Launches agent inside the VM with + `--dangerously-skip-permissions` by default. +- **Config**: Open / Balanced / Locked Down network presets at launch. No + per-role manifest. +- **Network policy**: Default-deny; preset levels control strictness. TUI + dashboard shows a live log of every outbound connection (allowed and + blocked) with point-and-click allow/block for hosts. +- **Credentials**: OS keychain + host-side proxy injection — API keys + never enter the VM. +- **Notable**: Best DX among microVM tools (one command, works like native + yolo Claude but inside a VM); branch mode creates a git worktree in + `.sbx/`. Network policy is preset-based, not role-declarative. +- **Maturity**: GA 2026. + +### Anthropic srt *(added 2026-07-18)* +- **Source**: https://github.com/anthropic-experimental/sandbox-runtime + (`@anthropic-ai/sandbox-runtime` on npm, `sandbox-runtime` on PyPI) +- **License**: Apache 2.0 (experimental). +- **Isolation**: OS-level only — Seatbelt (`sandbox-exec`) on macOS, + bubblewrap on Linux, WFP (Windows Filtering Platform) account-fenced on + Windows. **No container or VM.** Lowest overhead in the set. +- **Locality**: Local. +- **Agent integration**: Claude Code's sandboxed bash tool uses this + internally. Can wrap any arbitrary process (`srt `). Cloud + Claude Code sessions use full microVMs instead. +- **Config**: Programmatic per-invocation — allow/deny path lists for + filesystem; allow/denylist for network (HTTP proxy + SOCKS5). +- **Network policy**: Proxy-based filtering (HTTP + SOCKS5); domain + allowlist/denylist enforced at proxy layer. Custom proxy supported + (e.g. mitmproxy for inspection + audit). Processes that ignore proxy + env vars may bypass filtering on some platforms. +- **Notable**: Cross-platform (macOS/Linux/Windows); wraps any process, + not just agents; no role/manifest concept. Annotated as a research + preview — APIs may change. +- **Maturity**: Early research preview. + +## Governance / pre-action authorization layers + +These two tools don't provide VM or filesystem isolation; they intercept +tool calls before execution and evaluate them against a per-agent +declarative policy. They are the closest competitors on **agent-tailored +policy** and complement isolation sandboxes rather than substituting for +them. + +### Microsoft Agent Governance Toolkit (AGT) *(added 2026-07-18)* +- **Source**: https://github.com/microsoft/agent-governance-toolkit +- **License**: MIT (~3.3k stars, open-sourced April 2, 2026). +- **Isolation**: None (OS/VM). Execution rings (0–3, inspired by CPU + privilege levels) control what an agent can do at the framework layer. + MCP security gateway treats MCP traffic as an untrusted boundary. +- **Locality**: Embedded in the agent framework (Python, TypeScript, .NET, + Rust, Go; 20+ framework adapters). +- **Agent integration**: Framework-agnostic. Plugs into Semantic Kernel, + AutoGen, and others as a middleware layer. +- **Config**: YAML policy per agent — tools can be `allowed`, `denied`, + `sandboxed`, or routed through an `approval` step. Every action passes + through a governance gate checking: agent DID, trust score, risk tier, + requested tool, action type, and policy rules. +- **Network policy**: Not directly — operates at tool-call level. +- **Credentials**: Per-agent DID (Ed25519 decentralized identifier); agent + does not borrow a human's credentials. +- **Notable**: Dynamic trust score (0–1,000, behavioral decay) — + privilege follows observed behaviour, not just provisioning. Covers all + 10 OWASP Agentic Top 10 risks. Kill switch + SLO monitoring. Sub-ms + policy enforcement. +- **Maturity**: MIT, ~3.3k ⭐, v3.7.0 May 2026. + +### Open Agent Passport (OAP) *(added 2026-07-18)* +- **Source**: https://github.com/aporthq/aport-spec ; spec at + https://api.aport.io/spec/spec/oap/oap-spec.md/ ; arXiv 2603.20953 +- **License**: Open specification. +- **Isolation**: None. Pre-action hook only — intercepts tool calls + synchronously before execution, evaluates against a cloud-registry + declarative policy, fails closed. +- **Locality**: Local hook + cloud policy registry. +- **Agent integration**: Framework-agnostic; hook pattern. +- **Config**: Declarative policy rules in a cloud registry (evaluated in + order; first failing rule denies). Ed25519-signed, hash-chained audit + records per decision. +- **Network policy**: Not directly. +- **Notable**: 53ms median authorization decision (N=1,000). In an + adversarial testbed ($5,000 bounty, 1,151 sessions), social engineering + succeeded 74.6% of the time under a permissive policy; under a + restrictive OAP policy, 0% success across 879 attempts. Assumes + framework runtime is not compromised. +- **Maturity**: Specification + reference implementation, 2026. + ## Comparison table -| Axis | bot-bottle | endo-familiar | litterbox | agent-safehouse | matchlock | tilde.run | boxlite | microsandbox | smolmachines | CubeSandbox | -|---|---|---|---|---|---|---|---|---|---|---| -| Isolation | MicroVM per bottle default (Firecracker/KVM on Linux, Apple Container on macOS) + own egress DLP scanner; Docker legacy fallback, gVisor there if present | Object-capability (no OS isolation) | Podman + opt. Landlock | macOS `sandbox-exec` | MicroVM (Firecracker / Virt.fw) | Hosted container (unverified) | MicroVM (KVM / Hypervisor.fw) | MicroVM (libkrun) | MicroVM (libkrun / KVM) | MicroVM (RustVMM / KVM) | -| Local vs hosted | Local | Local | Local (Linux) | Local (macOS) | Local | Hosted SaaS | Local | Local | Local | Self-hosted (server/cluster) | -| Open source | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | MIT | No | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | -| Agent target | Claude Code | Generic (demo) | Generic | Multi-agent wrapper | Generic (+ Claude/OpenAI SDKs) | Claude focus | Generic | Claude + Cursor (MCP/Skills) | Generic (AGENTS.md) | E2B-compatible (platform builders) | -| Network policy | Default-deny via own egress scanner + per-bottle allowlist + content DLP + gitleaks on git push | Capability model only | Limited | Not addressed | Default-deny + allowlist + secret-injecting proxy | Default-deny + logging | Per-VM net (unverified) | Not documented | Off by default + allowlist | Default-deny allowlist + instant egress block + audit logs + per-sandbox tokens (eBPF) + credential vault | -| Parallel agents | Yes (one bottle per agent) | n/a | Not addressed | One at a time | Multiple VMs | Yes (dashboard) | SDK-level | SDK-level | Architectural | Yes (2,000+/host claimed) | -| Long-running posture | Persistent by default (named, supervised) | n/a (demo) | Session (up while in use) | Per-invocation | Ephemeral VM per run | Per-run (versioned) | Ephemeral + snapshot/fork | Ephemeral / on-demand | Named persistent by default | Ephemeral + auto pause/resume | -| DX: run Claude yolo-style | One command → interactive yolo Claude (`start `, `--dangerously-skip-permissions` default) | n/a (lib demo) | Wizard + build, then run claude inside (Linux only) | One-command wrapper (`safehouse claude --dangerously-skip-permissions`) | CLI: run a cmd in a VM (not a Claude wrapper) | Hosted (`tilde exec`), not local-native | SDK code required (build the run yourself) | CLI/MCP: sandbox-as-a-tool for the agent, not a wrapper around it | SSH into a named machine, run claude there | Stand up a cluster + drive via E2B SDK | -| Config | JSON manifest (bottles + agents) | Programmatic refs | CLI wizard | Profile files / shell fns | CLI / SDK | DSL + CLI + SDK | SDK | CLI / SDK / MCP | TOML Smolfile | E2B-compatible SDK | -| Maturity | Active May 2026 | Research (2022+) | Early (~66 ⭐) | Active (~1.4k ⭐) | Experimental (~574 ⭐) | Private preview | YC, ~4.7k ⭐ | YC, ~6k ⭐, beta | ~3.1k ⭐ | Tencent, prod, ~10.4k ⭐ | +*Isolation/sandbox tools only. AGT and OAP are governance layers — see their per-project notes above.* + +| Axis | bot-bottle | endo-familiar | litterbox | agent-safehouse | matchlock | tilde.run | boxlite | microsandbox | smolmachines | CubeSandbox | Cleanroom | container-use | Docker sbx | Anthropic srt | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| Isolation | MicroVM per bottle default (Firecracker/KVM on Linux, Apple Container on macOS) + own egress DLP scanner; Docker legacy fallback, gVisor there if present | Object-capability (no OS isolation) | Podman + opt. Landlock | macOS `sandbox-exec` | MicroVM (Firecracker / Virt.fw) | Hosted container (unverified) | MicroVM (KVM / Hypervisor.fw) | MicroVM (libkrun) | MicroVM (libkrun / KVM) | MicroVM (RustVMM / KVM) | MicroVM (Firecracker / Virt.fw) | Docker container + git worktree | MicroVM (proprietary) | OS-level (Seatbelt / bubblewrap / WFP) — no container | +| Local vs hosted | Local | Local | Local (Linux) | Local (macOS) | Local | Hosted SaaS | Local | Local | Local | Self-hosted (server/cluster) | Self-hosted server | Local | Local | Local | +| Open source | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | MIT | No | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | Proprietary | Apache 2.0 (experimental) | +| Agent target | Claude Code | Generic (demo) | Generic | Multi-agent wrapper | Generic (+ Claude/OpenAI SDKs) | Claude focus | Generic | Claude + Cursor (MCP/Skills) | Generic (AGENTS.md) | E2B-compatible (platform builders) | CI / generic process | Claude Code, Cursor, Windsurf (MCP) | Claude Code, Codex, Gemini CLI, Copilot, Kiro | Claude Code (and any process) | +| Network policy | Default-deny via own egress scanner + per-bottle allowlist + content DLP + gitleaks on git push | Capability model only | Limited | Not addressed | Default-deny + allowlist + secret-injecting proxy | Default-deny + logging | Per-VM net (unverified) | Not documented | Off by default + allowlist | Default-deny allowlist + instant egress block + audit logs + per-sandbox tokens (eBPF) + credential vault | Default-deny + per-repo host allowlist (cleanroom.yaml) | Not addressed | Default-deny; Open / Balanced / Locked Down presets; live TUI network panel | Proxy-based allowlist/denylist (HTTP + SOCKS5); custom proxy supported | +| Parallel agents | Yes (one bottle per agent) | n/a | Not addressed | One at a time | Multiple VMs | Yes (dashboard) | SDK-level | SDK-level | Architectural | Yes (2,000+/host claimed) | Yes (server model) | Yes (per-agent containers + worktrees) | Yes | Yes | +| Long-running posture | Persistent by default (named, supervised) | n/a (demo) | Session (up while in use) | Per-invocation | Ephemeral VM per run | Per-run (versioned) | Ephemeral + snapshot/fork | Ephemeral / on-demand | Named persistent by default | Ephemeral + auto pause/resume | Per-run + suspend/resume | Per-agent container (ephemeral) | Per-session; branch mode creates git worktree in .sbx/ | Per-invocation | +| DX: run Claude yolo-style | One command → interactive yolo Claude (`start `, `--dangerously-skip-permissions` default) | n/a (lib demo) | Wizard + build, then run claude inside (Linux only) | One-command wrapper (`safehouse claude --dangerously-skip-permissions`) | CLI: run a cmd in a VM (not a Claude wrapper) | Hosted (`tilde exec`), not local-native | SDK code required (build the run yourself) | CLI/MCP: sandbox-as-a-tool for the agent, not a wrapper around it | SSH into a named machine, run claude there | Stand up a cluster + drive via E2B SDK | CI-oriented, not a Claude wrapper | MCP server: `claude mcp add container-use -- container-use stdio` | One command: `sbx` wraps claude with `--dangerously-skip-permissions` default | Library/wrapper, not a standalone CLI | +| Config | JSON manifest (bottles + agents) | Programmatic refs | CLI wizard | Profile files / shell fns | CLI / SDK | DSL + CLI + SDK | SDK | CLI / SDK / MCP | TOML Smolfile | E2B-compatible SDK | cleanroom.yaml in repo | None (no policy config) | Preset levels at launch | Programmatic per-invocation (allow/deny lists) | +| Agent-tailored policy | Yes — bottle/agent split; declarative per-role egress + credentials; composable via `extends:` | Partial — capability model scopes per-agent, but no declarative role manifest | No | Partial — per-agent profile files (Seatbelt); no egress | No | Yes — per-agent DSL RBAC (allow/deny/approve per action/repo/agent) | No | No | No | No — per-sandbox SDK config, not role-scoped | Partial — per-repo cleanroom.yaml, not per-role | No | No — network presets only | No | +| Maturity | Active July 2026 | Research (2022+) | Early (~66 ⭐) | Active (~1.4k ⭐) | Experimental (~574 ⭐) | Private preview | YC, ~4.7k ⭐ | YC, ~6k ⭐, beta | ~3.1k ⭐ | Tencent, prod, ~10.4k ⭐ | Active (Buildkite product) | Early development | GA 2026 | Early research preview | ## What's closest, what's different @@ -233,11 +382,41 @@ existing OS primitive, low-dep. The split is the isolation primitive — bot-bottle now defaults to a VM per bottle (Firecracker microVM on KVM Linux, Apple Container on macOS) with its own DLP-scanning egress proxy, keeping Docker only as a legacy fallback; agent-safehouse uses -`sandbox-exec`; litterbox -uses Podman + Landlock. matchlock and smolmachines are close on *both* the -policy side (default-deny net, per-host allowlist) and — now that -bot-bottle has moved off containers-by-default — the microVM isolation -primitive. +`sandbox-exec`; litterbox uses Podman + Landlock. matchlock and +smolmachines are close on *both* the policy side (default-deny net, +per-host allowlist) and — now that bot-bottle has moved off +containers-by-default — the microVM isolation primitive. + +**New closest on agent-tailored policy.** Two governance tools are the +direct competitors on the "coarse-grained sandbox" axis. **tilde.run** +has had per-agent DSL RBAC since its launch (though it's hosted SaaS). +**Microsoft AGT** is the most serious new entrant: per-agent DID +identity, YAML policy that can allow/deny/sandbox/approve individual tool +calls per agent, and a dynamic behavioural trust score. It operates at +the framework tool-call layer, not the network layer — so it's +complementary to bot-bottle's network/filesystem isolation rather than a +direct substitute, but on the "does this sandbox know what this agent is +for?" question it is the most complete answer in the field. OAP's +pre-action hook pattern achieves similar goals with cryptographic audit +and a 0% adversarial-attack success rate under a restrictive policy. + +**New closest on DX.** **Docker sbx** is the first tool in this set that +matches bot-bottle on the "one command, dangerously-skip-permissions safe +by default" DX bar, at microVM isolation strength, with host-side +credential injection. It is proprietary, preset-based (not role- +declarative), and cloud-agent-specific, but it directly competes on the +UX proposition. agent-safehouse was the previous DX peer; Docker sbx +materially raises the bar. + +**New closest on repo-scoped policy.** **Cleanroom** (Buildkite) is the +first tool to combine microVM isolation with a declarative egress policy +file — though the policy lives in the repo being sandboxed +(`cleanroom.yaml`), not in an agent-role manifest. That makes it per- +repo rather than per-role: the same Cleanroom config applies to any +agent running in that repo. The distinction matters for bot-bottle's +use case (one developer running multiple agent *roles* with different +egress footprints), but for CI/CD use cases Cleanroom is a direct +alternative. **Solving a different problem.** tilde.run is hosted SaaS for team / production agent pipelines with data-versioned rollback — explicitly @@ -409,3 +588,57 @@ Why it matters anyway: and per-sandbox egress tokens (eBPF virtual switch vs. bot-bottle's mitmproxy egress proxy) before the next iteration of bot-bottle's in-flight-secret feature — see borrowable idea #2 above. + +## Addendum 2026-07-18 (second pass) — agent-tailored policy landscape + +The second-pass question was: how novel is bot-bottle's per-agent, +role-tailored sandbox relative to the expanded field? + +**The short answer:** on the isolation + network + role-tailoring +combination, bot-bottle remains the only tool in this set. On +role-tailored *policy at the tool-call level*, Microsoft AGT and OAP are +the most complete answers, but they don't provide isolation; they +complement rather than substitute. + +**The competitive picture by axis:** + +- *Agent-tailored egress (declarative, per-role)* — bot-bottle and + tilde.run. Cleanroom is per-repo, not per-role. Everyone else is + per-session or not addressed. +- *Agent-tailored tool-call policy (declarative, per-agent identity)* — + Microsoft AGT (YAML policy + DID identity + trust score), OAP + (declarative policy rules + cryptographic audit). Neither provides + network/filesystem isolation. +- *Composable policy (role overlays)* — bot-bottle (`extends:`). No + other tool surveyed supports composable role-policy inheritance. +- *Isolation + DX (one-command safe yolo)* — bot-bottle and Docker sbx. + Docker sbx is proprietary, preset-based, and cloud-agent-specific; + it's the first DX-class competitor at microVM isolation strength. + +**What the HN "coarse-grained" complaint maps to:** The complaint is +that a VM isolates the filesystem but doesn't know if the agent +*should* be sending an email. bot-bottle's bottle/agent split is a +structural answer to this: the bottle manifest declares exactly what +the role can reach, and the sandbox enforces it at the network layer. +Microsoft AGT is the most complete answer at the semantic/tool-call +layer. The gap both leave open is *intent classification* — knowing +whether a permitted action is consistent with the agent's actual task. +See `hn-agent-safety-discourse-july-2026.md` for the blast-radius +analysis. + +**Borrowable from new tools:** + +- **Microsoft AGT's trust-score decay** — privilege that reflects + observed behaviour rather than static provisioning. Applied to + bot-bottle: a bottle that has triggered DLP alerts or supervise holds + could auto-downgrade its network preset, or flag the session for + closer review. Fits the existing supervise-server architecture. +- **Docker sbx's live network TUI** — real-time per-session view of + allowed and blocked outbound connections with point-and-click + allow/block. `cli.py supervise` is the right surface; adding a + live-connections panel would directly address the "I can't see what + the agent is doing" gap without any backend changes. +- **OAP's cryptographic audit chain** — Ed25519-signed, hash-chained + audit records. Currently bot-bottle logs egress decisions but doesn't + chain them. A tamper-evident audit record per session would be useful + for the compliance use case the CubeSandbox positioning targets. diff --git a/docs/research/hn-agent-safety-discourse-july-2026.md b/docs/research/hn-agent-safety-discourse-july-2026.md index ff6fd5b..4e329d9 100644 --- a/docs/research/hn-agent-safety-discourse-july-2026.md +++ b/docs/research/hn-agent-safety-discourse-july-2026.md @@ -204,6 +204,43 @@ MCP STDIO server from within the agent is still sandboxed by the VM, and any outbound calls from that server must pass the egress allowlist and outbound DLP scanner. +**Per-agent role tailoring (the "coarse-grained sandbox" complaint)** + +The Feb 2026 HN thread that argued "sandboxes are too coarse-grained" +was pointing at a real gap: a VM isolates the filesystem but doesn't +know whether an agent *should* be sending email or calling an external +API. bot-bottle's bottle/agent split is a structural answer at the +network layer — the bottle manifest declares exactly what each role can +reach (which hosts, which paths, which HTTP methods), and the egress +scanner enforces it. A `gitea-dev` bottle that only lists +`gitea.dideric.is` and `api.anthropic.com` structurally cannot send +email or reach AWS, not because the model was told not to, but because +those routes don't exist. + +The `extends:` composition model means provider-level policy (the Claude +auth route) lives in one base bottle and role-specific overlays are +stacked on top — no duplication, and changing the base propagates to all +derived roles. + +Competitive position on this axis (from `agent-sandbox-landscape.md`): + +| Tool | Agent-tailored policy | +|---|---| +| **bot-bottle** | Yes — declarative per-role manifest; `extends:` composition; egress + credentials scoped to role | +| **tilde.run** | Yes — per-agent DSL RBAC (allow/deny/approve per action/repo/agent), but hosted SaaS | +| **Microsoft AGT** | Yes — YAML policy + per-agent DID + trust score, but tool-call level only (no network isolation) | +| **OAP** | Yes — declarative pre-action policy + cryptographic audit, but no isolation | +| **Cleanroom** | Partial — per-repo `cleanroom.yaml`, not per-role | +| **Docker sbx** | No — network presets only | +| **Anthropic srt** | No — programmatic per-invocation | +| **matchlock / smolmachines / microsandbox** | No | +| **agent-safehouse** | Partial — per-agent Seatbelt profiles; no egress | + +Two takeaways: bot-bottle and tilde.run are the only isolation tools +with declarative role-tailored policy; Microsoft AGT and OAP are the +closest competitors on role-tailoring but operate at the tool-call layer +without network/filesystem isolation — complementary, not substitutes. + **Outbound exfiltration (any injection class)** Whatever triggers the agent — README injection, Agentjacking, MCP From 015ff52eda1de0ce0ed6a9e6ba0e31efe059b2a7 Mon Sep 17 00:00:00 2001 From: didericis Date: Sat, 18 Jul 2026 15:27:29 -0400 Subject: [PATCH 2/3] fix(cli): exempt backend command from DB migration gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `backend setup/status/teardown` manage host prerequisites only and never open the store, but the CLI dispatcher ran the schema-migration gate before every command. On a non-TTY runner the gate's `Migrate now? [y/N]` prompt reads EOF and refuses, so `backend status --backend=firecracker` exits 1 — breaking the Firecracker CI preflight on any host without a pre-migrated DB. Exempt `backend` from the gate; store-touching commands stay gated. Fixes #419 Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX --- bot_bottle/cli/__init__.py | 9 +++++- tests/unit/test_cli_dispatch.py | 55 +++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 1 deletion(-) diff --git a/bot_bottle/cli/__init__.py b/bot_bottle/cli/__init__.py index 1dcb525..ae421fc 100644 --- a/bot_bottle/cli/__init__.py +++ b/bot_bottle/cli/__init__.py @@ -38,6 +38,13 @@ COMMANDS = { "supervise": cmd_supervise, } +# Commands that manage host prerequisites (or are otherwise store-free) and +# must run before — or without — a migrated DB. `backend` provisions/probes +# the host (TAP pool, /dev/kvm, firecracker) and never opens the store, so +# gating it on the schema breaks preflight on a fresh CI runner where stdin +# isn't a TTY and the migration prompt can't be answered. +NO_MIGRATION_COMMANDS = frozenset({"backend"}) + def usage() -> None: sys.stderr.write(f"usage: {PROG} [args...]\n\n") @@ -80,7 +87,7 @@ def main(argv: list[str] | None = None) -> int: usage() die(f"unknown command: {command}") mgr = StoreManager.instance() - if not mgr.is_migrated(): + if command not in NO_MIGRATION_COMMANDS and not mgr.is_migrated(): sys.stderr.write("bot-bottle: database schema is out of date\n") sys.stderr.write("Migrate now? [y/N] ") sys.stderr.flush() diff --git a/tests/unit/test_cli_dispatch.py b/tests/unit/test_cli_dispatch.py index 64cf655..2981f73 100644 --- a/tests/unit/test_cli_dispatch.py +++ b/tests/unit/test_cli_dispatch.py @@ -95,5 +95,60 @@ class TestMainDispatch(unittest.TestCase): self.assertEqual(130, main(["x"])) +class TestMigrationGate(unittest.TestCase): + """The dispatcher's schema-migration gate (cli/__init__.py).""" + + def setUp(self) -> None: + # Force the "schema out of date" branch for every test here. + patcher = patch.object(StoreManager, "is_migrated", return_value=False) + patcher.start() + self.addCleanup(patcher.stop) + self.addCleanup(StoreManager.reset) + + def test_store_command_blocks_when_stdin_cannot_confirm(self) -> None: + # Non-TTY stdin at EOF (as in CI): the [y/N] prompt reads "" and the + # command is refused rather than migrating silently. + ran: list[bool] = [] + + def handler(_rest: list[str]) -> int: + ran.append(True) + return 0 + + with patch.dict(climod.COMMANDS, {"list": handler}), \ + patch("sys.stdin", io.StringIO("")), \ + patch("sys.stderr", io.StringIO()): + self.assertEqual(1, main(["list"])) + self.assertEqual([], ran, "gated command must not dispatch") + + def test_backend_command_skips_gate(self) -> None: + # `backend` provisions/probes the host and never opens the store, so + # it must run even on an unmigrated DB with unanswerable stdin. + ran: list[bool] = [] + + def handler(_rest: list[str]) -> int: + ran.append(True) + return 0 + + with patch.dict(climod.COMMANDS, {"backend": handler}), \ + patch("sys.stdin", io.StringIO("")), \ + patch("sys.stderr", io.StringIO()): + self.assertEqual(0, main(["backend", "status"])) + self.assertEqual([True], ran, "exempt command must dispatch") + + def test_store_command_migrates_on_confirmation(self) -> None: + migrated: list[bool] = [] + + def handler(_rest: list[str]) -> int: + return 0 + + with patch.dict(climod.COMMANDS, {"list": handler}), \ + patch.object(StoreManager, "migrate", + side_effect=lambda: migrated.append(True)), \ + patch("sys.stdin", io.StringIO("y\n")), \ + patch("sys.stderr", io.StringIO()): + self.assertEqual(0, main(["list"])) + self.assertEqual([True], migrated, "confirmed gate must migrate") + + if __name__ == "__main__": unittest.main() From 5b359fe8d233cf3cd8d76ba3fbb8005d5e15a202 Mon Sep 17 00:00:00 2001 From: didericis Date: Sat, 18 Jul 2026 16:45:15 -0400 Subject: [PATCH 3/3] fix(git-gate): scan $new --not --all for every push, not $old..$new MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pre-receive hook scanned existing-branch updates with the delta range $old..$new. On a rebase / non-fast-forward force-push onto an advanced main, $old is no longer an ancestor of $new, so $old..$new expands to all of main's new history — including the deliberate sandbox-escape gitleaks fixtures — and the push is rejected on commits that belong to main, not the branch. Unify the range on `$new --not --all` for every non-delete push (this is the deferred open question from PRD 0028, which already applied it to new refs for #106). It scans only the commits the push introduces and is security-equivalent: the bare repo's refs come only from trusted upstream mirror-fetch and gitleaks-gated pushes, so an excluded commit is already-upstream or already-scanned. It is also more correct for non-fast-forward pushes, where $old..$new can skip commits off the direct path. Fixes #421 Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX --- bot_bottle/git_gate_render.py | 30 ++++++++++++++++++------------ tests/unit/test_git_gate.py | 16 +++++++++------- 2 files changed, 27 insertions(+), 19 deletions(-) diff --git a/bot_bottle/git_gate_render.py b/bot_bottle/git_gate_render.py index a7a93e8..2a2378c 100644 --- a/bot_bottle/git_gate_render.py +++ b/bot_bottle/git_gate_render.py @@ -419,18 +419,24 @@ PY while IFS=' ' read -r old new ref; do [ -z "$ref" ] && continue [ "$new" = "$zero" ] && continue - if [ "$old" = "$zero" ]; then - # New ref: scan only the commits this push introduces — those - # reachable from $new but not from any ref the gate already has. - # Everything already on the gate arrived via upstream mirror-fetch - # or a previously gitleaks-scanned push, so it's already-upstream - # or already-scanned; re-scanning it (the old `$new` full-ancestry - # range) only resurfaces historical findings and blocks every new - # branch. See PRD 0028 / issue #106. - log_opts="$new --not --all" - else - log_opts="$old..$new" - fi + # Scan only the commits this push introduces — those reachable from + # $new but not from any ref the gate already has. Everything already + # on the gate arrived via upstream mirror-fetch or a previously + # gitleaks-scanned push, so it's already-upstream or already-scanned; + # re-scanning it only resurfaces historical fixture findings. + # + # Applies to both new refs and updates. The old existing-branch range + # `$old..$new` walks commits reachable from the new tip but not the + # *old branch tip*: on a rebase/force-push onto a freshly-advanced + # main that pulls in all of main's new history (incl. the deliberate + # sandbox-escape gitleaks fixtures), blocking the push. `--not --all` + # excludes anything already on the gate regardless of ancestry, so it + # is also correct for non-fast-forward pushes (a rebase can skip + # commits off the direct path). Security-equivalent per PRD 0028's + # analysis: the bare repo's refs come only from trusted upstream + # mirror-fetch or gitleaks-gated pushes. + # See PRD 0028 (open question) / issues #106, #346. + log_opts="$new --not --all" echo "git-gate: gitleaks scanning $ref ($log_opts)" >&2 if ! gitleaks git --log-opts="$log_opts" --no-banner --redact 1>&2; then echo "git-gate: gitleaks rejected push to $ref" >&2 diff --git a/tests/unit/test_git_gate.py b/tests/unit/test_git_gate.py index 2f0acf4..cca39c5 100644 --- a/tests/unit/test_git_gate.py +++ b/tests/unit/test_git_gate.py @@ -168,16 +168,18 @@ class TestHookRender(unittest.TestCase): # Stdin is buffered to a tempfile so both phases can re-read. self.assertIn("refs_file=$(mktemp)", hook) - def test_new_ref_scan_scoped_to_incoming_commits(self): - # A new branch (old=all-zeros) must scan only commits new to the - # gate, not the full ancestry — otherwise historical findings - # block every new-branch push (PRD 0028 / issue #106). + def test_scan_scoped_to_incoming_commits(self): + # Every non-delete push scans only commits new to the gate, not + # the full ancestry and not the `$old..$new` delta — otherwise + # historical fixtures block new-branch pushes (PRD 0028 / #106) + # and a rebase/force-push onto an advanced main drags in main's + # history incl. the sandbox-escape fixtures (#346). hook = git_gate_render_hook() self.assertIn('log_opts="$new --not --all"', hook) - # The old over-broad full-ancestry range must be gone. + # Neither the full-ancestry range nor the ancestry-blind delta + # range may survive. self.assertNotIn('log_opts="$new"', hook) - # Existing-branch delta scan is unchanged. - self.assertIn('log_opts="$old..$new"', hook) + self.assertNotIn('log_opts="$old..$new"', hook) def test_forward_ssh_is_non_interactive_and_bounded(self): # No prompt (BatchMode) and a connect timeout, so an unreachable