Commit Graph

80 Commits

Author SHA1 Message Date
didericis 496608fc25 fix: correct harness defects found running the matrix
First full run on the KVM host surfaced three harness bugs and two stale
image URLs; all fixed here.

Harness bugs:
- Liveness probe used `bot-bottle --version`, which the CLI does not implement
  (unknown args die non-zero) — so every SUCCESSFUL install was misreported as
  "no runnable entry point". Switched to `bot-bottle --help`, which exits 0
  before any DB/migration/network work.
- cmd_down never removed serial.log, so its rmdir failed and every run left an
  orphan scratch dir behind. Added serial.log to the cleanup.
- The teardown trap was armed AFTER cmd_up, but cmd_up's wait_for_ssh can fail
  with QEMU already running (a guest that never opens SSH) — leaking the VM.
  Arm the trap before cmd_up.

Stale image URLs:
- Fedora 41 is EOL and 404s; bumped to Fedora 44 (44-1.7).
- Alpine bumped to 3.21.7; its cloud images ship a .sha512 only (this verifier
  is sha256), so SUM_URL is now empty (skip) with a note.

Validated on delphi (QEMU 11.0.2): ubuntu/fedora/arch pass BOTH test and
test-ready. Alpine (cloud-init seed not applied → no SSH) and NixOS (no
upstream cloud qcow2) remain blocked on image provisioning, documented in the
research note as follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 218f29cb05 refactor: adopt macOS test/test-ready convention for variants
Replaces the BB_TEST_PREREQS env toggle with the two subcommands the macOS
harness established, so the two repos read the same:

  test        A bare host — prerequisites NOT set up (the default state of a
              stock cloud image). Exercises install.sh's own prerequisite-guard
              logic. SOUND (PASS) when install.sh either installs cleanly or
              declines with one of its own recognized, actionable errors; a
              crash or unrecognized failure fails.
  test-ready  Prerequisites satisfied — the harness installs python3/git/pipx
              first, then runs install.sh, which must actually land: entry point
              runnable, doctor reporting a usable python and config.

Structure now mirrors scripts/macos-install-test.sh: a shared _test_cycle
driving up -> (prereqs) -> run -> verdict -> down, thin cmd_test / cmd_test_ready
wrappers setting _STEPS / _PASS_CLAIM / _REQUIRE_INSTALL, a standalone `prereqs`
subcommand, and a _test_teardown that prints the PASS/FAIL claim.

The doctor check is now the macOS-style classifier rather than a bare exit-code
read: a Traceback is an install defect (fail), a missing `ok: python:` /
`ok: config:` line is a fail, and a not-ready backend is reported, not fatal
(BB_TEST_REQUIRE_BACKEND=1 makes it fatal for a nested-virt host). This is where
Linux diverges from macOS test-ready — a plain VM has no nested KVM for
Firecracker and the harness doesn't provision the Docker backend, so backend
readiness is never the Linux criterion.

test-all now runs every distro × {test, test-ready}. Research note updated.

Validated with `bash -n` and `shellcheck`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 1518f73de5 feat: add bare-host install variant to the Linux harness
Splits the Linux clean-install harness into two variants, selected with
BB_TEST_PREREQS:

  with    (default) — install python3 + git + pipx first, then install.sh.
                      The ready-host happy path; PASS = install.sh exits 0 and
                      leaves a runnable bot-bottle entry point.
  without           — run install.sh on the BARE cloud image. Exercises
                      install.sh's own prerequisite-guard logic (python gate,
                      git-for-git-specs gate, pipx/pip PEP-668 handling). PASS =
                      install.sh either fully succeeds OR declines with one of
                      its own recognized, actionable prerequisite errors; a
                      crash or unrecognized failure is a FAIL.

cmd_run now captures install.sh's exit code + full output (install.rc /
install.log) instead of aborting on non-zero, so the verdict step applies the
variant's criterion. The old assert_installed is replaced by a quiet
entry_point_runnable helper plus classify_outcome, which matches the bare-host
declines against the exact die() messages install.sh prints. doctor still runs
for visibility, but only when an entry point exists.

test-all now runs the full matrix -- every distro x both variants -- each cell
in its own throwaway VM, and can be pinned to one variant via BB_TEST_PREREQS.
Research note updated to document both variants and their pass criteria.

Validated with `bash -n` and `shellcheck`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 9d82535390 feat: add Linux clean-install test harness
Adds scripts/linux-install-test.sh, a throwaway-VM harness that exercises
install.sh the way a brand-new user would across a Linux distro matrix
(Ubuntu, Fedora, Arch, Alpine, NixOS), plus the research note
docs/research/testing-clean-install-on-linux.md that motivates the approach.

This is the Linux counterpart to the macOS clean-install harness. On Linux
the boundary of choice flips from a throwaway user account to a disposable
KVM VM: it's a genuine kernel + userland + package-manager boundary that
wipes to nothing on teardown, and a single harness can swap distro cloud
images to cover the several package-management regimes Linux fragments into.
The host already needs KVM for the Firecracker backend, so a per-run,
copy-on-write VM (qemu-img create -b base) is cheap here -- the equivalent
of `docker run --rm`, for a whole machine.

Per run the harness caches one read-only base image, boots a throwaway
overlay via QEMU/KVM with user-mode networking (no root, no bridge), injects
an ephemeral SSH key + passwordless login through a cloud-init seed ISO,
installs the distro's prerequisites (python3 + git + pipx), pipes THIS
checkout's install.sh into the guest exactly as `curl ... | sh` would, and
asserts the CLI installed cleanly. The overlay is deleted on teardown, so
even the OS-level prerequisites are wiped -- unlike a throwaway user, the
reset is total.

Subcommands mirror the macOS harness (up/run/status/down/test) plus test-all
(the matrix) and ssh (an interactive guest shell). test arms an EXIT/INT/TERM
trap the moment the VM exists, so a failure or Ctrl-C still tears it down.

Scope is installer correctness, not runtime: there is no nested KVM/Docker in
the VM, so `bot-bottle doctor` correctly reports every backend not-ready and
exits non-zero by design. The pass criterion is therefore install.sh exiting
0, the bot-bottle entry point being present on the fresh user's PATH, and
`bot-bottle --version` running -- not a green doctor. doctor's output is still
printed so a real installer regression (broken shim, import error) stays
visible.

Validated with `bash -n` and `shellcheck`. Runtime is host-only (needs
/dev/kvm, qemu, cloud-localds), so like the macOS harness it isn't exercised
by the Linux PR CI. The cloud-image URLs in the DISTRO table are the one
place to bump when a distro cuts a newer build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis cd9f023f3d docs: name bot-bottle, not ./cli.py, in every instruction
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Successful in 59s
test / integration-docker (pull_request) Successful in 1m6s
test / coverage (push) Successful in 21s
test / image-input-builds (push) Successful in 44s
test / image-input-builds (pull_request) Successful in 1m15s
test / unit (push) Successful in 57s
test / coverage (pull_request) Successful in 20s
lint / lint (push) Successful in 1m3s
Update Quality Badges / update-badges (push) Successful in 1m12s
test / integration-docker (push) Successful in 1m0s
`doctor` told users to run `./cli.py backend setup`. cli.py is a four-line
wrapper at the repo root that calls bot_bottle.cli:main, and it does not ship —
[tool.setuptools.packages.find] includes only bot_bottle*, so anyone who
installed rather than cloned has no such file. Confirmed against a real
install: the user's tree contains exactly one executable, `bot-bottle`, and no
cli.py anywhere, while doctor recommended `./cli.py` three times. Invisible in
development, where ./cli.py works fine from a checkout, which is why only a
clean-install test surfaced it.

The two entry points are the same code, so the fix is to name the one that
always exists. 141 replacements across 45 files: the runtime messages that
caused this, plus README, docs, PRDs, research notes and test prose, so nothing
teaches the invocation a user cannot run.

scripts/demo.sh is deliberately untouched — it *executes* ./cli.py from a
checkout, where that is the correct and available path.

Verified end to end: a sandbox install now reports
"Run: bot-bottle backend setup --backend=firecracker", and bot-bottle is on
that user's PATH. Unit suite unchanged against the pre-existing baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 10:20:17 -04:00
didericis 5c499d290b docs: record what the passing runs established
test / image-input-builds (pull_request) Successful in 47s
test / unit (pull_request) Successful in 59s
test / integration-docker (pull_request) Failing after 2m50s
test / coverage (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 12m32s
prd-number-check / require-numbered-prds (pull_request) Failing after 12m45s
Both variants now pass, and the with-prerequisites run settled several things
that were guesses before:

* The Apple `container` service is per-user, confirmed directly rather than
  inferred: the throwaway account's appRoot is its own
  (/Users/bbtest/Library/Application Support/com.apple.container/), and
  starting it left the admin's service running and its agents intact.
* Setting up a new account is two steps, not one. The guest kernel lives in
  that same per-user app root, so a fresh account has none and
  `container system start` prompts to download it — and only prompts, since
  the flags default to asking. Headless callers need
  --enable-kernel-install.
* Neither step is performed by `bot-bottle backend setup
  --backend=macos-container`, which checks and then defers to `container
  system start`. So the real path for a new account is
  `container system start --enable-kernel-install`.
* Any of it requires entering the user's launchd domain with `launchctl
  asuser`, because the apiserver is a per-user agent reached over XPC.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 10:12:21 -04:00
didericis 71c0f8ed88 feat(harness): add test-ready, the with-prerequisites variant
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
tracker-policy-pr / check-pr (pull_request) Successful in 21s
test / image-input-builds (pull_request) Successful in 48s
test / integration-docker (pull_request) Successful in 1m4s
test / unit (pull_request) Successful in 2m52s
test / coverage (pull_request) Failing after 20s
"Does the installer work" and "can a new user actually run a bottle" are
different questions, and collapsing them is what made the previous run
ambiguous. Split them into two cycles over the same throwaway account:

  test        a macOS system WITHOUT the prerequisites set up for this user,
              which is the default state of every new account since the Apple
              `container` service is per-user. Asserts the install is sound;
              reports backend readiness without failing on it, because
              install.sh provides no backend and cannot regress one.

  test-ready  the same system WITH them. Runs `container system start` for the
              throwaway user, then demands doctor go fully green, backend
              included — the end-to-end claim.

The new `prereqs` step runs `container system start` rather than
`bot-bottle backend setup --backend=macos-container`, because that subcommand
does not start anything: it checks, then tells you to run `container system
start` yourself (backend/macos_container/setup.py, "no privileged host setup
required"). The harness runs what the product actually asks for.

test-ready sets BB_TEST_REQUIRE_BACKEND, so it demands exactly what `test`
merely reports. Both share one cycle function; the step count and the PASS
claim are the only differences beyond the extra step, so neither variant can
drift from the other's teardown or dirty-account guarantees.

Rig covers the distinguishing case directly: identical host state where `test`
passes on install soundness and `test-ready` starts the service and reaches a
green backend, plus the service-start failure and an install failure under
test-ready. 22 scenarios, all passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:51:35 -04:00
didericis 556217ae7b fix(harness): judge the install separately from backend readiness
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
test / image-input-builds (pull_request) Successful in 46s
test / integration-docker (pull_request) Successful in 1m3s
test / unit (pull_request) Successful in 2m58s
test / coverage (pull_request) Failing after 18s
tracker-policy-pr / check-pr (pull_request) Failing after 14m1s
The first run against the branch's own code confirmed the doctor fix — no
traceback, the firecracker probes now report "TAP pool: 0/8" instead of dying
on PermissionError — and then failed for a reason the installer cannot fix.

The Apple `container` service is per-USER. `container system status` reports an
appRoot under ~/Library/Application Support/com.apple.container/, and bbtest
sees "container system service: NOT running" while the creating admin's is
running. A brand-new account therefore has no backend until it runs
`container system start` once, doctor rightly fails, and `test` would be
permanently red for host state that install.sh neither creates nor can regress.

So stop treating doctor's exit code as one verdict. `test` now fails when the
install is broken — no entry point, doctor crashed, or doctor could not report
a usable python and config dir — and reports backend readiness as a note.
BB_TEST_REQUIRE_BACKEND=1 restores the strict behaviour.

A traceback is checked for explicitly rather than inferred from the exit code,
because those are the same value. The PermissionError bug exited non-zero
exactly like a missing prerequisite does; only the traceback distinguishes a
defect from an environment fact, and that distinction is the whole point of
this change.

The research doc's footprint table had the service as a host-wide launchd
service that survives user deletion. It does not. The state lives in the home
and goes with it, so the reset is more complete than claimed — but the
corollary is that a fresh account cannot run a bottle until the service is
started for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:45:57 -04:00
didericis 8d0782d1c7 fix(install): install into a private venv when pipx is absent
The harness's second run hit the wall the first one predicted: a fresh account
has no pipx, so install.sh fell to `pip install --user`, and every Python a Mac
offers — Homebrew and python.org alike — is externally managed, so PEP 668
blocked it. That fallback was never a fallback on macOS; it was a dead end that
printed instructions.

Replace it with a venv at ~/.bot-bottle/venv (BOT_BOTTLE_VENV to move it),
with the console script symlinked into ~/.local/bin. PEP 668 does not apply
inside a venv, and venv is stdlib, so unlike pipx there is nothing to bootstrap
first. pipx stays the preferred path when present, so anyone already managing
their Python apps that way is unaffected — and the post-install PATH check now
asks pipx for PIPX_BIN_DIR instead of assuming ~/.local/bin.

Keeping the venv under ~/.bot-bottle rather than ~/.local/share means the whole
footprint stays in one directory, which is what lets the throwaway-account
teardown remain a complete reset.

This removes the PEP 668 pre-flight and the sysconfig user-scheme lookup, both
of which existed only to serve the --user path. Their tests go with them:

* `detects_externally_managed_python` asserted the check that is now moot;
  replaced by one asserting pipx is still preferred when present.
* `checks_pip_usable_before_fallback` pinned a pip probe that no longer runs;
  replaced by one asserting the venv's own pip does the install, since using
  the base interpreter's would install outside the venv.
* `resolves_user_scripts_dir_not_hardcoded` and
  `macos_user_scheme_is_not_dot_local_bin` guarded the ~/Library/Python
  scripts-dir lookup. Nothing installs there now. The surviving "don't
  hardcode" concern is pipx's bin dir, which has its own test.

Five tests are added for the new path: the venv fallback exists, no --user path
survives, the venv is under the config dir, venv creation failure names
python3-venv (Debian ships it separately), and the entry point is exposed
outside the venv.

Verified end to end in a sandbox HOME with a fresh-account PATH and no pipx:
venv built, package installed, symlink created, `doctor` reached and green
(python 3.14.5, macos-container ready), exit 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis d415581adc feat: add a one-shot test cycle to the macOS install harness
`test` chains up -> run -> status -> down, which is the loop you actually want
when verifying a clean install. Three things make it more than a convenience
wrapper:

* It refuses to start against an existing account. A reused home is not a clean
  install, so testing one silently would defeat the harness.
* It tears the account down from an EXIT/INT trap armed the moment the account
  exists, so a failed run — or a Ctrl-C mid-install — still leaves the machine
  clean. BB_TEST_KEEP=1 opts out to poke at a failure.
* Its verdict is stricter than the installer's. install.sh exits 0 when it
  finishes but `doctor` reports unmet prerequisites, so "the installer
  succeeded" is not a useful assertion; `test` fails if the install fails, if
  bot-bottle never reached the new user's PATH, or if doctor is unhappy. That
  meant giving cmd_status a real exit status instead of swallowing doctor's.

Also fixes two bugs in `run`'s installer staging, by removing the staging
entirely and feeding install.sh in on stdin:

* `mktemp /tmp/bb-install.XXXXXX.sh` did not do what it looks like. BSD mktemp
  only substitutes trailing Xs, so every run wrote the *same* predictable path,
  as root, mode 644, in a world-writable directory.
* The cleanup only ran on the normal and failure returns, so an interrupted run
  leaked the file.

Piping on stdin sidesteps both: root opens the redirect before sudo drops
privileges, so the mode-700 home that motivated the staging is a non-issue,
there is no file to leak, and `sh -s` matches the documented `curl … | sh`
shape more closely than executing a staged copy did.

The command-level `exit 1`s become `return 1` so the steps compose under the
trap, and the "next, run this" hints are suppressed inside `test`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis-claude 27b9ba5247 feat: add macOS clean-install test harness
Add scripts/macos-install-test.sh, a throwaway-user harness for exercising
install.sh the way a brand-new user would on macOS, plus the research note
that motivates the approach.

The harness has up/run/status/down/deep-reset subcommands. Because install.sh
writes only to the user home (pipx venv, ~/.bot-bottle, a PATH line) and never
installs the backend, deleting the account is a complete, deterministic reset
of the install surface. A disposable macOS VM can't stand in on M1/M2: the
Apple `container` backend needs Virtualization.framework, and running it inside
a guest VM requires nested virtualization (M3+ only), so a throwaway user is
the only way to reach the real host backend from a clean $HOME.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 09:38:20 -04:00
didericis-codex 82e1a353bd docs: analyze Sandbox Agent SDK architecture 2026-07-27 03:53:02 +00:00
didericis-codex a6e1aebda1 docs: update agent sandbox competitor landscape 2026-07-27 03:44:41 +00:00
didericis-codex 9c06702b32 docs(prd): number merged placeholder documents
prd-number / assign-numbers (push) Failing after 25s
lint / lint (push) Successful in 4m7s
Update Quality Badges / update-badges (push) Failing after 13m37s
test / coverage (push) Has been skipped
test / integration-docker (push) Failing after 13m51s
test / integration-firecracker (push) Failing after 13m48s
test / integration-macos (push) Failing after 13m54s
test / unit (push) Failing after 13m53s
test / publish-infra (push) Has been skipped
2026-07-26 06:06:04 +00:00
didericis d0b595828f feat(macos): run containers inside a bottle via guest-local podman
Adds the `nested_containers` bottle flag. On the macOS backend it starts
a rootless podman service inside the bottle and exposes its
Docker-compatible API socket, so the agent still runs `docker` and
`docker compose`. No host daemon socket is mounted and the guest gains
no capabilities; backends that cannot run a guest-local engine reject
the flag in the shared prepare template rather than ignoring it.

Podman rather than rootless Docker because Apple Container's capability
bounding set omits CAP_SYS_ADMIN, which the kernel requires to write a
multi-range uid_map via newuidmap. With no subordinate UID range podman
falls back to a single-UID self-mapping an unprivileged process may
write itself, so the image build strips /etc/subuid and /etc/subgid
entries rather than adding them.

That mapping is also why nested containers are not an isolation layer:
root inside one is the agent user outside it. They are a build/test
convenience; the bottle remains the boundary.

Ports the spike branch onto main, renaming docker_access — it implied
Docker and granted access to nothing on the host — and drops podman
from the derived layer now that every built-in image ships it (#451).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 22:54:26 -04:00
didericis-claude 182a28d724 docs(research): firecracker image remote store
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 40s
tracker-policy-pr / check-pr (pull_request) Successful in 6s
test / integration-firecracker (pull_request) Successful in 3m33s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 35s
test / unit (push) Successful in 39s
Update Quality Badges / update-badges (push) Successful in 47s
lint / lint (push) Successful in 51s
test / integration-firecracker (push) Successful in 5m8s
test / coverage (push) Successful in 44s
test / publish-infra (push) Successful in 1m53s
Investigates storage candidates, retention policy design, and
image-size reduction techniques in response to the PR #459 comment
on a secure remote store for committed Firecracker snapshots.
2026-07-21 22:47:44 -04:00
didericis-codex af1690ab22 docs(research): consolidate agent sandbox landscape 2026-07-20 14:36:40 +00:00
didericis-claude 09debcf4f0 docs(research): add Agent Safehouse HN launch and Apple Container 1.0 stable
- agent-sandbox-landscape.md: update agent-safehouse star count (~1.4k →
  ~1.8k), add HN thread #47301085 link and key discussion notes (creator
  framing, Simon Willison observation, top community quote on sandboxing
  being THE major challenge), note Apple Container 1.0 stable (Jun 9 2026)
- hn-agent-safety-discourse-july-2026.md: add Agent Safehouse March 12
  launch to the sandboxing boom section as context that prefigures the
  June–July shift in community tone

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-20 13:51:52 +00:00
didericis-claude 4302678f3e docs(research): expand sandbox landscape with 6 new tools; add agent-tailored policy axis
tracker-policy-pr / check-pr (pull_request) Failing after 6s
Isolation tools added: Cleanroom (Buildkite), container-use (Dagger),
Docker sbx, Anthropic srt.

Governance/pre-action layers added as a separate section: Microsoft
Agent Governance Toolkit (per-agent DID + YAML policy + trust score),
Open Agent Passport (declarative policy + cryptographic audit).

Comparison table: 14 → 14 columns; new Agent-tailored policy row added.
Second addendum covers competitive position on role-tailoring, Docker
sbx as new DX-class competitor, and borrowable ideas (trust-score decay,
live network TUI, cryptographic audit chain).

Discourse note: adds Per-agent role tailoring to "What it covers well"
with competitive comparison table across 9 tools.
2026-07-18 19:11:14 +00:00
didericis-claude 727eafe0f9 docs(research): clarify prompt injection framing and blast-radius risks
Collapse "trusted-channel data injection" into prompt injection
throughout — the trusted channel is a delivery vector, not a distinct
attack class. Add explicit inbound/outbound orthogonality framing.
Replace the two redundant "weaker" bullets with a single prompt
injection section and a new blast-radius breakdown covering work
product corruption, malicious commits past gitleaks, exfiltration
through allowlisted channels, and dependency-install injection.
2026-07-18 18:13:33 +00:00
didericis-claude 1ec114b6d7 docs(research): survey HN agent safety discourse June-July 2026
Covers the CVE cascade (DuneSlide, CVE-2026-39861, MCP STDIO injection),
Agentjacking and README-injection attack classes, community opinion
clusters, and a frank assessment of where bot-bottle covers or falls
short against each issue.
2026-07-18 18:08:40 +00:00
didericis aa44feea02 docs(research): note on malicious-commit scanning at the git-gate + paid-feature analysis
Adds a research note on whether/how to scan for malicious code (not just
secrets) in commits pushed through the git-gate, and whether the semantic
(LLM) layer is a defensible paid feature.

Verdict: no scanner reliably detects malicious code (undecidable +
adversarial), so the frame is raise-cost + cover-the-obvious + human-gate
the dangerous. Ranked layers: dependency/supply-chain scanning (Socket/OSV/
GuardDog) > heuristic/obfuscation (Semgrep-on-diff) > risk-based human
gating via the existing supervise plane > best-effort LLM diff-review.
Fast scanners inline in the synchronous pre-receive; heavy analysis async.

Monetization: the paid unit is the governed git-egress review bundle
(managed semantic review + web-console human-review flow + RBAC + audit +
cross-run policy), not the raw scanner — which stays OSS like gitleaks.
Extends the egress audit+custody wedge to code artifacts; the supervise
console generalizes across all proposal types (egress, gitleaks, commit
review). Sell the workflow, not the detector's accuracy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 07:00:22 -04:00
didericis f2e2572a40 docs(research): add DX axis — "run Claude yolo-style" — to the sandbox landscape
Adds a "DX: run Claude yolo-style" row to the comparison table plus a note
framing developer experience as a differentiator. The field splits into
wrappers-around-the-agent (bot-bottle, agent-safehouse — one command, the
agent just runs, `--dangerously-skip-permissions` on by default with the
sandbox as the guardrail) vs libraries/services (boxlite, microsandbox,
CubeSandbox, E2B — you wire the agent in via SDK/cluster). agent-safehouse
is the only DX peer, but it's macOS-only Seatbelt with no egress story.
"As easy as native yolo, but actually sandboxed" is the defensible line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:41:26 -04:00
didericis 7069fa225d docs(research): add long-running posture axis to the sandbox landscape
Adds a "Long-running posture" row to the comparison table and an addendum
note contrasting the two models: E2B and CubeSandbox are ephemeral-per-task
(5-min default timeout, tier-capped continuous runtime, duration via
pause/resume + reconnect-by-id), while bot-bottle bottles are persistent,
named, and supervised by default. For agents that run for hours/days this
posture difference matters more than the isolation primitive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:26 -04:00
didericis aa224c4381 docs(research): add CubeSandbox to the sandbox landscape; fix stale bot-bottle self-description
Adds CubeSandbox (Tencent Cloud, Apache 2.0, RustVMM/KVM microVM) to the
agent-sandbox landscape survey: per-project note, comparison-table column,
and a dated addendum on what it means for positioning. CubeSandbox is the
first surveyed project to bundle a connection-level egress allowlist +
audit + in-flight credential custody, but it does NOT do content DLP on
authorized channels — that plus the orchestration layer is where
bot-bottle stays distinctive.

Also corrects two stale self-descriptions the survey (2026-05-11) baked
in and I'd propagated:
- Default isolation is now a VM per bottle (Firecracker microVM on KVM
  Linux, Apple Container on macOS); Docker is only the legacy fallback,
  per _default_backend_name(). Was described as Docker-by-default.
- Outbound DLP is bot-bottle's own mitmproxy egress scanner + gitleaks on
  git push, not pipelock (removed). All references updated; a note
  records the change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:12 -04:00
didericis c69642e568 feat(macos): consolidated per-host gateway for the Apple backend (PRD 0070)
Re-enables the macos-container backend on the shared per-host orchestrator +
gateway, replacing the per-bottle companion container removed in #385. This is
the last backend in PRD 0070's roadmap.

Apple Container 1.0.0 forced three departures from the docker shape, each
verified against the live CLI (findings recorded in the networking spike):

- No `--ip`. The address is DHCP-assigned and knowable only once the container
  runs, so the order inverts: gateway up -> run agent -> read its address ->
  register. The identity token is minted by registration and therefore cannot
  be in the agent's run-time env; it rides the proxy URL applied at
  `container exec` time (bare `--env` names keep it off argv).
- No container DNS. The gateway can only be handed the control plane's IP, so
  the orchestrator starts first and the gateway is pointed at its address.
- No `network connect`. Networks are fixed at run time, so the shared host-only
  network is created up front; per-bottle networks would restart the gateway
  on every launch and defeat the consolidation.

The agent runs with `--cap-drop CAP_NET_RAW`: Apple grants NET_RAW by default,
which would let an agent forge a neighbour's source address on the shared
segment. NET_ADMIN is already absent, so this closes the source-address half of
PRD 0070's attribution invariant.

Verified end-to-end on real Apple Container 1.0.0: both images build, the
control plane comes up healthy, the gateway reaches it by IP, and a registered
agent gets 200 for a host in its routes and 403 for one outside them. Bring-up
is idempotent — a second launch does not churn the singletons.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 01:27:40 -04:00
didericis 4d89ffce8b docs(research): add local/internal-reach edge to the OneCLI comparison
Self-hosting means the agent runs inside your own network boundary (homelab,
corporate LAN, Tailnet), so scoping it to internal resources is just an
egress-route line — a reach advantage distinct from the isolation ones, which a
cloud-first credential broker / agent product can't match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhiafsABCr46bu3oHUm7wa
2026-07-14 01:48:48 -04:00
didericis 822cff48ad docs(research): mark credential-proxy note as implemented (agents get placeholders)
The note still described the pre-gateway state as "today" — raw provider
tokens env-injected into the agent, readable via printenv — which is no longer
true and was actively misleading. Add a dated Status banner and flag the stale
"today's wiring" / "recommended path forward" passages: the agent now holds only
a placeholder (e.g. CLAUDE_CODE_OAUTH_TOKEN=egress-placeholder) and the real
token is injected as the upstream Authorization header by the existing
pipelock/mitmproxy egress sidecar (EgressRoute auth_scheme + token_ref, one MITM
not two), generalized across Claude/Codex/Pi and git-host tokens. Reasoning
preserved per the research-note convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhiafsABCr46bu3oHUm7wa
2026-07-14 01:42:05 -04:00
didericis ce440fabc4 docs(research): refresh OneCLI competitor entry + product verdict
OneCLI (onecli.sh) was already tracked in the credential-proxy landscape but
the entry was stale (May 2026). Correct it: it uses the phantom-token pattern
this note recommends (not "Bitwarden integration"), and it's now GA, Rust,
YC-backed (~2.5k stars, 300k+ downloads). Add build-vs-adopt + competitor
commentary, and a product-side entry in the containerized-claude landscape with
a verdict on how close a competitor it is and where bot-bottle's edge lies
(isolation as the product, fleet/manifest layer, self-hosted trust posture).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhiafsABCr46bu3oHUm7wa
2026-07-14 01:23:32 -04:00
didericis 049103ac99 docs: mark smolmachines-specific PRDs + research as superseded
test / unit (pull_request) Successful in 50s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 1m4s
Add a "Superseded (2026-07-11)" banner and flip Status: Active →
Superseded on the 8 smolmachines-specific docs (7 PRDs + the VM-backend
research). Bodies are left intact as a historical record; the banner
points at the removal commit and the landscape doc so a reader isn't
misled into thinking the backend still exists.

Incidental mentions in other PRDs (git-gate, cred-proxy, print-parity,
etc.) are left as-is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 19:09:43 -04:00
didericis-claude 814c7338a1 docs(research): update landscape doc — SuperHQ, multi-backend, multi-provider, in-flight directions
- Broaden scope from "Claude Code in Docker" to "AI coding agents in isolated sandboxes"
- Add SuperHQ (superhq.ai, v0.4.4) as a new adjacent-competitor entry: macOS-only
  microVM desktop app, overlaps on isolation/credential-proxy/multi-provider but has
  no manifest layer and no audit logging
- Note SuperHQ's user-voiced audit gap (Brian Cheong, Dunialabs.io) and that
  bot-bottle already covers it
- Update differentiation list to reflect three backends (Docker, Apple container,
  smolmachines) and three built-in providers (Claude Code, Codex, Pi) plus plugin system
- Add in-flight directions for forge-native dispatch (#317) and paid web control plane
  (#327) with honest framing: lifecycle concept is not novel vs. cloud services (Devin,
  Copilot Workspace); differentiation is self-hosted + manifest-driven + stronger isolation
2026-07-09 18:55:15 +00:00
didericis 80eca740d6 docs(research): replace unsourced "20% malicious skills" with cited empirical figures
The "~20% of ClawHub skills malicious" claim had no traceable source and
is contradicted by the empirical literature. Replace with the Jan 2026
large-scale study (98,380-skill snapshot: 157 confirmed malicious, ~71%
credential harvesters, exfiltration overwhelmingly naive) and add the
arXiv citation. The corrected figures still support the supply-chain
threat point and are defensible under scrutiny.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 09:32:19 -04:00
didericis 7644da4280 docs: add Apple Container transparent egress spike 2026-06-10 22:36:55 -04:00
didericis 13e4af421d docs: add Apple Container networking spike 2026-06-10 22:36:55 -04:00
didericis-claude 8c0a9c5bc6 docs: rename PRD 0053 to PRD 0052
Renames docs/prds/0053-egress-dlp-addon.md to 0052-egress-dlp-addon.md
and updates all references in the documentation.
2026-06-06 16:27:04 +00:00
didericis-claude 5265e25f9b docs: address PR #196 review; update research decisions and PRD
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 41s
Research doc: close open questions with decisions from review — hard
cutover on path_allowlist, drop glob (regex sufficient), stick with
Gateway API OR semantics for headers, case-insensitive method names.

PRD 0053: adopt Gateway API HTTPRoute match vocabulary (paths, methods,
headers) as the route schema replacement for path_allowlist. Add
MatchEntry / PathMatch / HeaderMatch types to EgressRoute design; cite
the route matching research doc; fold match restructure into chunk 1
alongside the dlp block.
2026-06-05 00:52:57 +00:00
didericis-claude 035ed430ba docs: research on YAML route matching formats (paths, headers, methods)
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 46s
2026-06-05 00:41:19 +00:00
didericis c94a2542bd docs: evaluate CaMeL prompt injection framework for integration
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 36s
test / integration (push) Successful in 54s
Add analysis of Google DeepMind's CaMeL (arXiv:2503.18813), which
prevents prompt injections architecturally rather than detecting them.

Key findings:
- CaMeL operates at the agent execution layer (P-LLM/Q-LLM split +
  capability-based data flow tracking), not the network layer
- Not a replacement for pipelock/DLP — different threat surface
- Not viable today: research artifact, requires agent rearchitecture,
  doubles LLM costs, 7% utility loss on AgentDojo
- Worth watching: its capability model could complement bot-bottle's
  network controls if it matures into production software

Also clarifies pipelock's actual detection capabilities (no prompt
injection detection) and adds naive detector sketch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-04 14:13:32 -04:00
didericis e6b3cd1824 docs: remove time estimates and add LLM-based detection analysis
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 45s
- Remove all time estimates (2-3 weeks, 1-2 weeks, etc.)
- Add detailed analysis of using LLM for prompt injection detection
- Survey existing models (none purpose-built for this)
- Sketch DistilBERT fine-tuning approach (~67MB quantized)
- Analyze latency/footprint tradeoffs (50-150ms vs. <5ms for patterns)
- Recommend pattern-based Phase 2, with LLM as optional Phase 2b
- Include code sketch of LLM detector with timeout fallback
- List open questions for LLM deployment

Conclusion: Patterns are faster/simpler for now; LLM only if patterns
miss sophisticated attacks in production.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-04 14:02:59 -04:00
didericis 49f77f2d1e docs: accommodate PR feedback on detector architecture
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 50s
Per feedback from PR 192:

- Restructure around outbound_detectors (requests to upstream) and
  inbound_detectors (responses from upstream)
- Rename to 'secret exfiltration' detection for Phase 1
- Add 'known_secrets' detector for provisioned credentials
- Make scanning enabled by default per detector type
- Clarify that multiple encodings of secrets should be checked

Phase 1 now focuses on preventing outbound credential leaks.
Phase 2 handles inbound prompt injection attacks.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-04 13:54:46 -04:00
didericis d3c2d9e8f6 docs: research document on DLP alternatives to pipelock
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 47s
Investigates replacing pipelock with a custom mitmproxy-based DLP addon
that supports per-route configuration, response-specific rules, and
AI-specific threat detection (tokens, prompt injection).

Recommends building the addon in-repo to align with bot-bottle's
per-route design model and keep security logic auditable.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-04 13:21:42 -04:00
didericis-claude d1556f4659 docs(research): local ollama deployment, harness selection, and model sizing
test / unit (push) Successful in 41s
test / integration (push) Successful in 48s
2026-06-03 21:37:55 -04:00
didericis-claude 06eed5b236 docs(research): gitea webhook agent dispatch and PR session continuity
test / unit (push) Successful in 38s
test / integration (push) Successful in 51s
Research note covering how to spawn bot-bottle agents from Gitea
webhook events and reuse the same session (bottle identity + Claude
session ID) across an entire PR lifecycle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 21:37:40 -04:00
didericis ae1531835d docs: drop "forge" jargon for concrete Gitea wording
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 53s
test / unit (push) Successful in 36s
test / integration (push) Successful in 57s
We use Gitea, not an abstract forge. Reword the docs added in this
branch: "forge thread" -> "Gitea thread", and the research note's
generic "forge" -> "Gitea" / "hosting provider" as context demands,
keeping its portability argument coherent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-28 23:05:02 -04:00
didericis 5c5f576df0 docs(research): add README describing research notes
Document what research notes are (opinionated investigations of a
question/design space), their unnumbered kebab-case naming, and their
loose verdict-first shape — explicitly freeform, not a template. Point
the AGENTS.md research line at it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-28 23:05:02 -04:00
didericis c840182d12 docs(research): issue tracking vs in-repo decision history
Analyze tracking feature requests in Gitea against the project's
in-repo PRDs/research notes, given the goal of keeping decision
history portable and not provider-locked. Recommends demoting issues
to an ephemeral inbox and reifying durable rationale into the repo.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-28 23:05:02 -04:00
didericis 7b4c1cd091 docs: drop "forge" jargon for concrete wording
test / unit (pull_request) Successful in 26s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 28s
test / integration (push) Successful in 42s
We use Gitea, not an abstract forge. Reword the pre-existing research
and PRD docs: the generic "Forge-API gate"/"forge tokens" become
"Git-host-API gate"/"Git-host tokens" (the gate still spans Gitea /
GitHub / GitLab), "Git/forge history" -> "Git/Gitea history", and the
KNOWN_FORGE_HOSTS / forge: manifest-field examples -> KNOWN_GIT_HOSTS
/ git_host:. Meaning preserved; only the word "forge" is dropped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-28 22:57:20 -04:00
didericis-codex 18e3b62b72 docs: rename CLAUDE.md to AGENTS.md and rebrand provider-agnostic
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 40s
test / unit (push) Successful in 31s
test / integration (push) Successful in 44s
Delete CLAUDE.md in favor of AGENTS.md as the orientation doc, rebrand
the project from Codex-bottle to provider-agnostic bot-bottle, and
repoint every CLAUDE.md reference across PRDs, research notes, the
implementer agent example, and the yaml_subset comment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-28 20:36:47 -04:00
didericis-codex cdb1870b1c docs(agent): clarify claude oauth env
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 43s
2026-05-28 18:20:09 -04:00
didericis-codex c08b09dc9f refactor!: rename project to bot-bottle
Assisted-by: Codex
2026-05-28 17:56:14 -04:00