Split launch() into named per-step helpers, replace time.sleep(1.5) with
a readiness poll, and file-lock loopback alias allocation. Addresses the
three actionable items from the #117 hotspot review of smolmachines/launch.py.
Expands scope to cover both remaining egress hotspot tasks from #117:
- Replaces the named-helper design with a flat provisioned-wins merge
(provider routes own their hosts; manifest fills gaps; no upgrade or
conflict-detection logic needed).
- Adds _route_to_yaml_fields as the single authoritative EgressRoute→Route
mapping to prevent silent type drift between host and addon.
- Notes that the mitmproxy pure-function split is already clean (decide +
is_git_push_request) and requires no structural change.
Revises the Design section to describe the implemented solution:
provisioned_env on AgentProvisionPlan rather than an intermediate
egress_resolve_token_values_with_provider function. Drops the old
sentinel/lazy-import design narrative.
Extracts the forward_host_credentials / CODEX_HOST_CREDENTIAL_TOKEN_REF
resolution block, currently copy-pasted in both docker and smolmachines
launch files, into a single shared function in bot_bottle/egress.py.
Closes#118. Found via #117 hotspot review.
Operators can now declare:
agent_provider:
template: claude
auth_token: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
and the provisioner injects a provider-owned api.anthropic.com egress
route (Bearer, tls_passthrough) rather than requiring a manually
declared route with the former claude_code_oauth role.
Changes:
- Add auth_token field to AgentProvider; validate claude-only.
- Remove claude_code_oauth from EGRESS_ROLES / PROVIDER_EGRESS_ROLES.
Manifests that declare the role now fail at parse time with "unknown
role" — the provisioner owns the route.
- agent_provision_plan: replace manifest_egress_routes/has_provider_auth
with auth_token; Claude branch injects the api.anthropic.com route,
placeholder env, and nonessential-traffic flags when auth_token is set.
- Add hidden_env_names: frozenset[str] to AgentProvisionPlan; Claude
branch populates it with CLAUDE_CODE_OAUTH_TOKEN.
- Remove auth_role from AgentProviderRuntime and placeholder_env_for().
- print_util.visible_agent_env_names: accept hidden_env_names from the
plan instead of dispatching on agent_provider_template.
- Both backends: drop manifest_egress_routes call, pass auth_token.
- PRD 0029 rescoped to cover both Codex and Claude provider auth.
Assisted-by: Claude Code
Debugging a live codex smolmachines bottle surfaced three independent
failures past the sign-in screen; fix each so forward_host_credentials
works end to end:
- codex_auth: dummy access/id tokens now inherit the *real* host token's
exp instead of now+1h. Codex (0.135) refreshes when its local token's
JWT exp lapses; with a placeholder refresh_token that refresh fails and
drops to the sign-in screen. Aligning exp tracks the real token's life.
- prepare: set CODEX_CA_CERTIFICATE to the agent CA bundle for codex
bottles. Codex is rustls and ignores the system store / NODE_EXTRA_CA_
CERTS; it reads CODEX_CA_CERTIFICATE (fallback SSL_CERT_FILE) for custom
roots across HTTPS + wss, so it must be pointed at the egress MITM CA or
injection can't work without tls_passthrough.
- pipelock: auto tls_passthrough the Codex API hosts when
forward_host_credentials is on. Egress injects the bearer before
pipelock, whose header DLP then flags the JWT ("request header contains
secret") and the retry storm trips its 429. passthrough host-gates the
CONNECT but skips decrypt+rescan of egress-owned auth. The auto-added
routes aren't in bottle.egress.routes, so the hosts are added explicitly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
git-gate's pre-receive scans the full ancestry of a new branch, so the
repo's historical test-fixture findings block every new-branch push
(issue #106). Scope the new-ref scan to incoming commits
(`$new --not --all`) with no loss of coverage, and harden the forward
ssh against hangs.
Refs #106
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record that we considered auto-generating an agent's system prompt from
its bottle's egress/git config (so it would know its access up front)
but opted to keep prompts operator-authored: we may want to withhold
that information from the agent directly, and the agent can infer its
access on its own regardless.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
We use Gitea, not an abstract forge. Reword the docs added in this
branch: "forge thread" -> "Gitea thread", and the research note's
generic "forge" -> "Gitea" / "hosting provider" as context demands,
keeping its portability argument coherent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document what research notes are (opinionated investigations of a
question/design space), their unnumbered kebab-case naming, and their
loose verdict-first shape — explicitly freeform, not a template. Point
the AGENTS.md research line at it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the one-line docs/INDEX.md (its directory pointers are covered
by docs/README.md's "when to write which document" table). Add
docs/prds/README.md documenting the PRD naming, Status lifecycle, and
section format. Repoint the AGENTS.md repository-layout list at the
new READMEs and add the decisions/ dir.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the document-type comparison out of docs/decisions/README.md
(where it only surfaced if you were already in the decisions dir) up
to a new docs/README.md, renamed "When to write which document".
Leave a pointer from the decisions README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per review on PR #97: an index that lists every ADR is a sync
burden. The files in docs/decisions/ are the index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an "Alternatives considered" section enumerating the design
options from issue #88 (duplicate bottles / agent-side bottle_config
/ bottle-side extends) and why extends won, so the PRD stands without
the forge thread. Repoint the two phrases that depended on the #88
comment thread at the new section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add docs/decisions/ with a convention README and back-fill two
decisions that previously had no in-repo home: merging PRs with
rebase (ADR 0001) and the agent-identity claimed-not-vouched trust
posture from PRD 0027 (ADR 0002). Point docs/INDEX.md at it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Analyze tracking feature requests in Gitea against the project's
in-repo PRDs/research notes, given the goal of keeping decision
history portable and not provider-locked. Recommends demoting issues
to an ephemeral inbox and reifying durable rationale into the repo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
We use Gitea, not an abstract forge. Reword the pre-existing research
and PRD docs: the generic "Forge-API gate"/"forge tokens" become
"Git-host-API gate"/"Git-host tokens" (the gate still spans Gitea /
GitHub / GitLab), "Git/forge history" -> "Git/Gitea history", and the
KNOWN_FORGE_HOSTS / forge: manifest-field examples -> KNOWN_GIT_HOSTS
/ git_host:. Meaning preserved; only the word "forge" is dropped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flip Status: Draft -> Active for the 23 PRDs whose work has shipped to
main (including 0027, now that PR #95 has merged). Leaves the
terminal-status PRDs unchanged: 0007 and 0010 (Superseded) and 0014
(Retargeted) were replaced, not shipped as-is.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lift git.user (name/email) to the agent layer with a per-field
overlay onto the referenced bottle, mirroring the extends: merge.
git.remotes stays bottle-only. Includes identity provenance in
preflight/info and an example collapse.
Refs #94
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete CLAUDE.md in favor of AGENTS.md as the orientation doc, rebrand
the project from Codex-bottle to provider-agnostic bot-bottle, and
repoint every CLAUDE.md reference across PRDs, research notes, the
implementer agent example, and the yaml_subset comment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Earlier commit framed this PR as "infrastructure landed, TSI
enforcement blocked on upstream smolvm 0.8.0." Found a clean
workaround that lets us enforce now.
Smolvm persists each machine's config (including
`allowed_cidrs`) as a JSON BLOB in
`~/Library/Application Support/smolvm/server/smolvm.db`,
`vms.data`. `machine create --allow-cidr X/32` silently writes
`allowed_cidrs: null` to that row when combined with `--from`,
but smolvm reads the row at `machine start` — so patching the
row between create and start sets the allowlist for real.
New `loopback_alias.force_allowlist(machine_name, cidrs)` opens
the SQLite DB, JSON-decodes the row, sets `allowed_cidrs`, and
writes back as BLOB (Text type silently corrupts smolvm's
later reads). launch.py calls it immediately after
`machine_create` and before `machine_start`.
Verified end-to-end on macOS / Docker Desktop:
VM allowlist after start: ["127.0.0.16/32"]
VM → 127.0.0.1:3000 → BLOCKED (Permission denied)
VM → 8.8.8.8:53 → BLOCKED (Permission denied)
VM → 127.0.0.16:<bundle> → CONNECTED
The DB-patch hack is correct only because smolvm reads
`allowed_cidrs` from the row at start time (not derived in-
process). When upstream honors `--allow-cidr` with `--from`,
the call becomes redundant — drop the call and the workaround
is gone.
Tests: 4 new for `force_allowlist` (BLOB round-trip; Linux
no-op; missing DB; missing row). Total 593 unit tests pass.
README + PRD updated to reflect the fix landed (no longer
"infrastructure pending upstream"). gitea#75 can close.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #76 originally claimed the per-bottle alias scoping closed
gitea#75 ("agent can reach host loopback"). Verified
empirically that's not actually true: `smolvm 0.8.0 machine
create --from <smolmachine> --net --allow-cidr X/32` silently
drops the allowlist (`agent.config.json` shows `allowed_cidrs:
null`, and the running VM reaches all of `127.0.0.0/8`
regardless).
So the alias-allocation + alias-bind infrastructure is correct
pre-work, but the actual TSI enforcement is blocked on an
upstream smolvm bug. README + PRD 0023 + the module docstring
get reworded to say so plainly. gitea#75 stays open.
Workarounds tried (all dead-ends):
- `machine update --allow-cidr` doesn't exist
- stop-edit-`agent.config.json`-restart fails (smolvm removes
the file on stop)
- `--smolfile` is mutually exclusive with `--from`
- `--image localhost:<port>/...` fails because smolvm's agent
process can't reach host loopback during pull
When upstream lands a fix, our existing code (alias allocation,
port-bind, --allow-cidr in launch) will scope correctly without
further changes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #74's Docker-Desktop fix routed the agent through
`127.0.0.1:<random>` loopback forwards, but TSI filters by IP
only — so the allowlist `127.0.0.1/32` let the agent VM reach
**any** host service on macOS loopback (postgres, dev servers,
other bottles' published ports, mDNSResponder, ...). Real
downgrade vs the docker backend's `--internal` network.
Resolution: per-bottle loopback alias.
- New `loopback_alias` module manages a pool of
`127.0.0.16` .. `127.0.0.31` on `lo0`. macOS only routes
`127.0.0.1` by default; the extras need `sudo ifconfig lo0
alias`. `ensure_pool()` lazily adds the missing entries via
one sudo prompt on first launch per reboot — aliases persist
on `lo0` until reboot, so subsequent launches skip the
prompt entirely.
- `allocate(slug)` picks the lowest-numbered unused alias by
inspecting running bundle containers' port-binding HostIps.
No on-disk reservation — docker is the source of truth.
- Bundle bringup binds published ports to the allocated alias
(`docker run -p <alias>::<port>`) instead of `127.0.0.1`.
- TSI allowlist becomes the alias's /32 — narrows reachability
to this bottle's bundle only.
- Linux native daemons share the host's network namespace;
`127.0.0.0/8` works without aliases, so the module no-ops on
non-Darwin and returns `127.0.0.1` from `allocate`.
Tracking issue closed: gitea/issues/75.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #74's Docker-Desktop pivot widened the smolmachines TSI
allowlist from `<bundle-ip>/32` to `127.0.0.1/32` (TSI can't
filter by port, and docker bridge IPs aren't reachable from
macOS networking). The agent VM can therefore reach any service
on macOS's loopback while the bottle is running — not just the
bundle's published ports.
README gets a "Smolmachines backend" subsection under Quickstart
spelling this out as a known v1 limitation. PRD 0023 grows a new
open question #8 with the proposed v2 fix (per-bottle loopback
alias + TSI allowlist scoped to that /32, via sudo
`ifconfig lo0 alias`).
Tracking issue: gitea.dideric.is/didericis/claude-bottle/issues/75.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the alpine:latest placeholder with a real claude-bottle
agent image, converted into a .smolmachine artifact via an
ephemeral local OCI registry.
Why the registry hop: smolvm pack create only accepts OCI registry
refs. Empirically it rejects docker-daemon://, oci-layout://,
docker-archive: tarballs, and every other transport tested — the
crane backend treats anything with a scheme prefix as a registry
hostname. To convert a locally-built docker image into a
.smolmachine we have to push it somewhere smolvm can pull from.
Smallest path: bring up registry:2.8.3 bound to 127.0.0.1:<random>,
docker tag + docker push into it, smolvm pack create --image
localhost:<port>/claude-bottle:<id>, tear down the registry.
The .smolmachine is cached under
~/.cache/claude-bottle/smolmachines/ keyed by the docker image ID
(first 16 hex chars of the sha256), so a Dockerfile change picks
up a new image ID and invalidates the cache. Unchanged rebuilds
skip the whole build → registry → pack pipeline.
This puts `docker build` in smolmachines prepare (the docker
backend defers it to launch). Necessary because pack_create needs
the image ID to derive the cache key, and prepare is the only
hook ahead of launch that runs once per slug.
Adds:
- claude_bottle/backend/docker/util.py: image_id / tag / push
helpers (thin docker CLI wrappers).
- claude_bottle/backend/smolmachines/local_registry.py:
ephemeral_registry() context manager; pins registry:2.8.3 by
digest, binds 127.0.0.1::5000 (loopback-only), force-removes on
exit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Chunk-1's empirical spike against smolvm 0.8.0 contradicted the
research note that motivated the gvproxy network design: smolvm
exposes no virtio-net-over-unixgram attachment. The first draft's
"why gvproxy, not TSI" argument turns out to apply only to
`--outbound-localhost-only`, not to TSI generally.
New design:
- Bundle (PRD 0024) runs on a dedicated per-bottle docker bridge
with a pinned IP. Smolfile sets `[network] allow_cidrs =
["<bundle-ip>/32"]` and nothing else. Agent can reach the bundle
and nothing else — host loopback, LAN, public internet directly
are all refused at the VMM (TSI) layer.
- Bind-address mitigation: egress binds 127.0.0.1:9099 inside the
bundle (pipelock-internal); pipelock / git-gate / supervise
bind 0.0.0.0 so the agent (across the TSI allowlist) can reach
them. This is the port-granularity TSI's IP-only allowlist
doesn't provide.
- Smolfile renderer rewritten in chunk 2 to smolvm 0.8.0's actual
schema (image / entrypoint / cmd / env / [network] allow_cidrs).
The chunk-1 renderer (name= / [[net]]= under the gvproxy
design) emits the wrong shape and will be replaced.
- Drop gvproxy + VZFileHandleNetworkDeviceAttachment + the
PyObjC fallback. Backend layout loses gvproxy_config.py,
gvproxy.py, vfkit_attach.py.
- Acceptance plan adds an egress-port-bypass probe in addition
to the localhost-reach probe.
- Chunks reshape: chunk 1 stays (renderer rewrite is part of
chunk 2's cost); chunk 2 covers VM lifecycle + bundle + new
Smolfile renderer; chunk 3 is the bundle bind-address change;
chunks 4-5 unchanged in spirit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Compose-up has owned per-container lifecycle since PRD 0018 ch3;
the .start() / .stop() methods on DockerPipelockProxy /
DockerEgress / DockerGitGate / DockerSupervise (and their
abstractmethod declarations in the four base ABCs) were already
documented as vestigial. With the bundle path in flight
(PRD 0024 ch2), they are truly dead — collapse to nothing.
Changes:
- Removed start/stop methods from the four DockerSidecar
classes. Plan dataclasses, image/path constants,
container-name helpers, and the .prepare() methods all stay
(the renderer + apply path still need them).
- Removed the matching @abstractmethod declarations in the
base ABCs so concrete subclasses don't have to stub them.
- launch.launch() and prepare.resolve_plan() no longer take
proxy/git_gate/egress/supervise instance parameters. backend.py
loses the four instance attributes it threaded through.
prepare.resolve_plan() instantiates the four classes itself
to call their .prepare() methods.
- Deleted four integration tests that only exercised the
removed lifecycle: test_pipelock_sidecar_smoke,
test_supervise_sidecar, test_git_gate_sidecar,
test_git_gate_mirror.
- Dropped the .stop-idempotency case in test_orphan_cleanup;
the network-cleanup cases stay (those test real production
code).
- Marked test_pipelock_apply @skip pending chunk 4 — its
bringup helper used .start; chunk 4 rewrites it with direct
`docker run`.
Dockerfile deletion deferred to chunk 5 (when the bundle flag
default flips) — the legacy compose path still needs
Dockerfile.{egress,git-gate,supervise} until then.
Net: 708 lines removed, 80 added.
533 unit tests + 27 integration tests passing (5 skipped: the
chunk-4-pending case + existing GITEA_ACTIONS guards).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reverses chunk 1's "any unexpected child death tears down the
rest" policy. New behavior: a daemon dying is logged but does
NOT initiate shutdown — the surviving daemons keep running and
whatever the dead one served starts failing visibly on the
agent side. The supervisor exits only when (a) it receives
SIGTERM/SIGINT, or (b) every child has died on its own.
Eventual design is restart-the-dead-daemon plus a notification
to the supervise sidecar so the operator sees the event
explicitly; this commit ships only the "log and leave alone"
half. PRD 0024 open question 1 updated to reflect the new
intent.
Tests updated: replaced "crash propagates exit code via
auto-teardown" with three cases that exercise the new policy
(crash without shutdown leaves survivors up, crash-then-signal
surfaces the nonzero code, all-children-die-unattended still
converges the loop).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace pipelock + egress + git-gate + supervise as four
separate containers with one bundle image
(claude-bottle-sidecars) running all four daemons under a small
stdlib Python init supervisor. Compose file collapses from five
services to two; same daemons, same ports, same protocols, one
container.
Sized: bundle image + init → renderer collapse (feature-flagged)
→ backend Python trim → integration sweep → flag removal.
Prerequisite for PRD 0023 chunk 3 (smolmachines backend reuses
the same bundle as its sole host-side sidecar container).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the four host-side sidecar processes (pipelock + egress +
git-gate + supervise) with a single bundled container per bottle,
defined in PRD 0024 and consumed here. egress is internal to the
bundle as pipelock's upstream; only pipelock, git-gate, and
supervise are externally addressable, and only when the bottle
uses them.
gvproxy port_forwards collapse from one-per-process to one-per-
external-port, all pointing into the one bundle container.
Sizing: chunk 3 becomes "sidecar bundle lifecycle" and depends
on PRD 0024 having landed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis-claude
didericis-codex
didericis