e26d459a975aae3c55bd782f85b868ad0e5b6131
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 didericis-claude
|
47eb56bd10 |
fix(smolmachines): use containerized crane to push, bypassing docker daemon's HTTPS preference
The previous fix (`host.docker.internal:<port>` for daemon-side push) still failed: Get "https://host.docker.internal:53958/v2/": http: server gave HTTP response to HTTPS client `host.docker.internal` is reachable from Docker Desktop's daemon VM but isn't in the daemon's default insecure-registries CIDRs (only `::1/128` and `127.0.0.0/8` are), so docker push tries HTTPS, hits a plain-HTTP registry, and refuses. The daemon.json fix (`"insecure-registries": ["host.docker.internal"]`) works but is a one-time manual step in Docker Desktop's UI — not something we can do for the user. Sidestep the daemon push entirely: 1. docker build (as before) — local layer cache makes no-change rebuilds cheap. 2. docker save the image to a per-digest tarball alongside the cached `.smolmachine`. 3. Start an ephemeral registry container on a per-session docker network, with `-p :5000` so the host can also reach it for the pack step. 4. docker run a one-shot crane container on the SAME network, mount the tarball, `crane push --insecure /img.tar <registry-container>:5000/...`. Container DNS resolves the registry on the network; `--insecure` forces plain HTTP. 5. `smolvm pack create --image localhost:<host port>/...` from the host. smolvm's bundled crane auto-falls-back to HTTP for localhost addresses, so no insecure-registries config is needed on that side. 6. Tear down everything; reap the tarball (registries hold the same bytes, no need to keep both around). Net effect: the docker daemon never does an HTTP/HTTPS-policy decision on our behalf. `docker push` is gone from the prepare path; `docker save`, `docker network create`, `docker run` (for registry + crane) replace it. Tested end-to-end on Docker Desktop / macOS: `_ensure_smolmachine ("claude-bottle:latest")` produces a 204MB `.smolmachine.smolmachine` artifact. Adds: - backend/docker/util.py:save() — thin docker save wrapper. - local_registry.crane_push_tarball() — one-shot crane run on the registry's network. - CRANE_IMAGE constant pinned by digest (gcr.io/go-containerregistry/crane@sha256:0ae17ecb...). Removes: - backend/docker/util.py:tag() / push() — unused without daemon push. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
didericis-claude
|
f4026ea3ae |
fix(smolmachines): docker push fails on Docker Desktop — daemon-side route differs from host loopback
`./cli.py start <agent>` under CLAUDE_BOTTLE_BACKEND=smolmachines died at `docker push localhost:<port>/claude-bottle:<id>` with `Get "http://localhost:<port>/v2/": context deadline exceeded`. Cause: chunk 4c bound the ephemeral registry to `127.0.0.1::5000` and used `localhost:<port>` as the only image-ref hostname. On Docker Desktop the daemon runs inside its own Linux VM — its `localhost` is the VM's loopback, not the host's, so the daemon cannot reach a registry bound to the host's 127.0.0.1. Fix: bind the registry to all interfaces (`-p :5000`) so it's reachable from both sides, and yield two endpoints: - `daemon_endpoint` — `host.docker.internal:<port>` on Docker Desktop (daemon-side hostname for the host VM gateway), `localhost:<port>` on a native Linux daemon that shares the host's network namespace. Used for `docker tag` + `docker push`. - `host_endpoint` — always `localhost:<port>`. Used for `smolvm pack create`, which runs as a host process. The registry stores images by repo+tag, so a push to `host.docker.internal:<port>/cb:<id>` and a pull from `localhost:<port>/cb:<id>` resolve to the same blob — the hostname in a ref is just routing. Detection uses `docker info --format '{{.OperatingSystem}}'`, which returns "Docker Desktop" on macOS/Windows Desktop and the host's OS name on native daemons. Trade-off: all-interface binding briefly publishes the registry on every interface (~5-10s during prepare). The pushed image is built from the public repo Dockerfile (no secrets), the port is random, and the window is short — acceptable for v1 of a personal dev tool. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
didericis-claude
|
1fa17d1822 |
feat(smolmachines): build agent image from repo Dockerfile (PRD 0023 chunk 4c)
Replaces the alpine:latest placeholder with a real claude-bottle agent image, converted into a .smolmachine artifact via an ephemeral local OCI registry. Why the registry hop: smolvm pack create only accepts OCI registry refs. Empirically it rejects docker-daemon://, oci-layout://, docker-archive: tarballs, and every other transport tested — the crane backend treats anything with a scheme prefix as a registry hostname. To convert a locally-built docker image into a .smolmachine we have to push it somewhere smolvm can pull from. Smallest path: bring up registry:2.8.3 bound to 127.0.0.1:<random>, docker tag + docker push into it, smolvm pack create --image localhost:<port>/claude-bottle:<id>, tear down the registry. The .smolmachine is cached under ~/.cache/claude-bottle/smolmachines/ keyed by the docker image ID (first 16 hex chars of the sha256), so a Dockerfile change picks up a new image ID and invalidates the cache. Unchanged rebuilds skip the whole build → registry → pack pipeline. This puts `docker build` in smolmachines prepare (the docker backend defers it to launch). Necessary because pack_create needs the image ID to derive the cache key, and prepare is the only hook ahead of launch that runs once per slug. Adds: - claude_bottle/backend/docker/util.py: image_id / tag / push helpers (thin docker CLI wrappers). - claude_bottle/backend/smolmachines/local_registry.py: ephemeral_registry() context manager; pins registry:2.8.3 by digest, binds 127.0.0.1::5000 (loopback-only), force-removes on exit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
didericis-claude
|
73dc0d4a40 |
refactor(sidecars): instantiate sidecar ABCs directly from any backend
The four sidecar prepare-time helpers (PipelockProxy, Egress, GitGate, Supervise) had docker-flavored subclasses that existed only as instantiation shims for ABCs that already had no abstract methods. PipelockProxy.prepare() reached for class-level CA path constants that were only defined on the docker subclass — so smolmachines had to import DockerPipelockProxy to render pipelock yaml, reaching across the backend boundary for what's actually a platform-neutral operation. This moves the universal in-container CA paths (PIPELOCK_CA_CERT_IN_CONTAINER / PIPELOCK_CA_KEY_IN_CONTAINER) to claude_bottle/pipelock.py, drops the class-attr indirection on the ABC, and deletes the four empty docker subclasses. Both backends now instantiate the ABCs directly; the docker-side modules keep the docker-flavored helpers (image pin, container naming, host CA mint) and re-export the moved pipelock constants for compat. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
didericis-claude
|
1dfc359141 |
feat(smolmachines): thread inner Plans + bundle daemons run (PRD 0023 chunk 4b)
Bundle daemons (pipelock, egress, optionally git-gate + supervise)
now actually start with their config files bind-mounted from the
inner Plans the docker backend already produces. Chunks 2d + 3
ran with daemons_csv="" so the bundle's init supervisor idled;
chunk 4b wires up the real path: agent → pipelock → egress →
internet (when routes declared) is now functional, modulo agent-
image gaps (claude-code / TLS-trust-store / git in the guest)
that chunk 4c addresses.
bottle_plan.py — added the four inner Plan fields:
proxy_plan: PipelockProxyPlan
git_gate_plan: GitGatePlan
egress_plan: EgressPlan
supervise_plan: SupervisePlan | None
Same shape the docker backend's plan uses. Docker-network-only
fields (internal_network, egress_network) stay at dataclass
defaults — the smolmachines bundle is on a per-bottle bridge
with a pinned IP, not docker's --internal + egress topology.
prepare.py — instantiates DockerPipelockProxy / DockerEgress /
DockerGitGate / DockerSupervise and calls their .prepare()
methods to write the per-bottle config files (pipelock.yaml,
routes.yaml, git-gate entrypoint/hooks, supervise queue dir)
under the per-bottle state dir. (The "Docker" prefix on the
class names is a misnomer here — .prepare() is platform-neutral,
inherited from each sidecar's ABC. A future cleanup could factor
the prepare logic out of the docker subpackage.)
launch.py — major rewrite:
- pipelock_tls_init at launch (always); egress_tls_init only
when the bottle declares routes (otherwise the CA files
aren't bind-mounted and openssl runs would be wasted).
- Inner Plans updated in place with launch-time CA paths +
EGRESS_UPSTREAM_PROXY = http://127.0.0.1:8888 (egress's
upstream is pipelock on the bundle's own loopback; same
container's network namespace).
- BundleLaunchSpec env + volumes built from the inner Plans:
pipelock.yaml + CA + key (always); egress routes + CAs +
upstream env + token-slot bare names (when routes); git-gate
entrypoint + hooks + per-upstream identity files (when
upstreams); supervise queue dir + env (when enabled).
- daemons_csv = ["egress", "pipelock"] + ["git-gate"] (if
upstreams) + ["supervise"] (if enabled).
- Token env values resolved from host env via
`egress_resolve_token_values` and threaded into the
docker-run subprocess env (bare-name -e entries in spec
inherit from there — values never land on argv).
Tests:
- 552 unit passing (no new unit cases; fixture updated to
populate the new plan fields).
- 5 integration cases passing locally (Darwin + smolvm + docker
+ not GITEA_ACTIONS):
* test_smoke_exec_echo — still works.
* test_localhost_reach_probe — host loopback still refused.
* test_egress_port_bypass_probe — <bundle-ip>:9099 still
refused, NOW WITH EGRESS ACTUALLY RUNNING (chunk 3's
127.0.0.1 bind-address is doing its job).
* test_prompt_file_lands_in_guest — still works.
* test_pipelock_answers_on_bundle_ip — NEW. From inside the
guest, wget to <bundle-ip>:8888 gets an HTTP response
(not "connection refused") — proves pipelock is actually
listening and the bind-mount + CA generation path works.
What's left in chunk 4:
- 4c: agent-image-conversion (claude-code + git + curl +
ca-certificates in the guest). Chunk 2d's alpine placeholder
stays for now.
- 4d: provision_ca + provision_git + provision_supervise once
the agent image has the required tools.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
didericis-claude
|
9e3b7e441e |
feat(smolmachines): provision_prompt + provision_skills (PRD 0023 chunk 4a)
First slice of chunk 4: implement the two provisioning methods
that don't depend on agent-image tooling beyond `cp` and
`mkdir`. provision_ca / provision_git / provision_supervise
land once the agent-image gap is solved (chunk 4b+) — they need
update-ca-certificates, git, and the claude binary respectively,
none of which the chunk-2d alpine placeholder provides.
What this PR ships:
- `claude_bottle/backend/smolmachines/provision/` subpackage
with `prompt.py` + `skills.py`. Each routes through
`smolvm.machine_cp` / `machine_exec`. provision_prompt mirrors
the docker contract (file always copied; return value drives
--append-system-prompt-file iff the agent has a non-empty
prompt). provision_skills mkdir + cp per skill, matching
the docker backend's loop.
- prepare.py now writes the prompt file under
agent_state_dir(slug) with the agent's `prompt` body, mode
0o600. The in-guest path is `/root/.claude-bottle-prompt.txt`
(alpine has no `node` user; will become `/home/node/...` once
the real claude-bottle image lands).
- launch.py calls `provision(plan, machine_name)` after
machine_start. The returned prompt path threads to
SmolmachinesBottle so exec_claude can add
--append-system-prompt-file when the agent has a prompt.
- backend.py: provision_prompt / provision_skills now real;
provision_git is a deliberate stub (waiting on the git-gate
inner Plan + git in the agent image). provision_supervise
stays the chunk-2d stub.
Tests:
- 7 new unit cases (test_smolmachines_provision.py): argv
shape (mocked smolvm.machine_cp / .machine_exec),
prompt return-value contract, no-op-with-no-skills,
CLAUDE_BOTTLE_GUEST_SKILLS_DIR override, fail-on-missing-skill.
- 1 new integration case in test_smolmachines_launch.py:
end-to-end verification that the prompt file lands in the
alpine guest at /root/.claude-bottle-prompt.txt with the
expected content (via `bottle.exec("cat ...")`). The smoke +
the two TSI probes stay green.
552 unit + 4 integration (Darwin+smolvm+docker gated) passing.
What's left in chunk 4:
- 4b: thread the inner Plans (PipelockProxyPlan / EgressPlan /
GitGatePlan / SupervisePlan) through prepare + launch so the
bundle daemons actually run (currently daemons_csv="").
- 4c: the agent-image-conversion gap — get claude-code + git +
curl + ca-certificates into the guest image (build a
.smolmachine via `pack create --from-vm` after manual setup,
or push the docker image to a registry smolvm can pull).
- 4d: provision_ca + provision_git + provision_supervise once
4b + 4c land.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
didericis-claude
|
9f65b137b9 |
feat(smolmachines): end-to-end launch + Bottle.exec + smoke + probes (PRD 0023 chunk 2d)
End-to-end launch flow for the smolmachines backend. Brings up
the per-bottle docker bridge + sidecar bundle, creates and
starts the smolvm guest pointed at the bundle's pinned IP via
TSI's `--allow-cidr <bundle-ip>/32`, yields a SmolmachinesBottle
handle that routes exec/cp through `smolvm machine exec / cp`,
tears everything down on context exit.
launch.py:
- ExitStack-managed: create_bundle_network → start_bundle →
machine_create → machine_start (each registered for reverse
teardown).
- daemons_csv="" for chunk 2d — bundle init logs "no daemons
selected" and idles. Real daemon bringup with inner-Plan-driven
env + volumes lands in chunk 4.
bottle.py:
- SmolmachinesBottle.exec → smolvm.machine_exec (captured).
- SmolmachinesBottle.exec_claude → direct subprocess.run with
inherited TTY for interactive sessions.
- SmolmachinesBottle.cp_in → smolvm.machine_cp.
Architecture pivots forced by smolvm 0.8.0's CLI shape:
1. `--from <smolmachine>` and `--smolfile <toml>` are MUTUALLY
EXCLUSIVE in smolvm 0.8.0. We need --from to avoid the
registry-pull race that bit us on machine_start (libkrun
agent's network attempt got refused by macOS with
"connect: permission denied" on IPv6). So Smolfile is dropped
entirely; per-bottle env + allow_cidrs flow as CLI flags
(`--allow-cidr CIDR`, `-e K=V`) directly to machine_create.
2. `smolvm pack create --image` doesn't pull from the local
docker daemon — only OCI registries via crane. The real
claude-bottle:latest image lives in the local docker daemon
and isn't reachable that way. Chunk 2d ships with an alpine
placeholder; the agent-image-conversion gap belongs to
chunk 4 (push the image to a registry, or smolvm grows a
docker-daemon transport).
Other changes:
- machine_create grew `image=` / `from_path=` / `allow_cidrs=`
/ `env=` kwargs; smolfile= dropped.
- bottle_plan: smolfile_path → agent_from_path + guest_env.
- prepare: pack_create against `alpine:latest`, cached under
~/.cache/claude-bottle/smolmachines/ keyed by image ref.
- Deleted smolfile.py + test_smolfile.py (dead code now).
Tests:
- Unit: 540 passing (smolvm wrapper grew 4 new flag forms; one
test renamed to reflect --from + --allow-cidr + -e combo).
- Integration: 3 new cases in tests/integration/
test_smolmachines_launch.py, gated on Darwin + smolvm on PATH
+ docker + not GITEA_ACTIONS:
* smoke: bottle.exec("echo hello-from-vm") round-trips with
the correct stdout + returncode.
* localhost-reach probe: agent dials 127.0.0.1:9 → connect
refused (TSI's <bundle-ip>/32 allowlist doesn't include
loopback). The regression test for the gap the PRD design
pivot was about.
* egress-port-bypass probe: agent dials <bundle-ip>:9099
(egress's port) → connect refused. Chunk 2d has no
daemons running so nothing's listening anyway; chunk 3
will preserve this property once egress is up but bound
to 127.0.0.1 inside the bundle.
End-to-end smoke + both probes green locally on macOS with
smolvm 0.8.0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
didericis-claude
|
c73d717f71 |
feat(smolmachines): rewrite Smolfile to smolvm 0.8.0 schema + drop gvproxy (PRD 0023 chunk 2a)
First sub-PR of chunk 2: rewrite the renderer chunk 1 shipped to match smolvm 0.8.0's actual Smolfile shape, delete the dead gvproxy renderer + its tests, simplify the prepare flow now that there's no gvproxy socket + no loopback-port allocation. Smolfile renderer: - Old shape (under the abandoned gvproxy design): name = ..., command = [...], [[net]] attachment = "unixgram", socket = "...". - New shape (smolvm 0.8.0): env = [...] (sorted K=V pairs), [network] allow_cidrs = ["<bundle-ip>/32"]. Nothing else. image / entrypoint / cmd come from the .smolmachine artifact built in chunk 2b; cpus / memory left at smolvm defaults. - Tests assert no leakage of TSI's --outbound-localhost-only or the old gvproxy/unixgram keys. util.py: - smolmachines_gvproxy_subnet → smolmachines_bundle_subnet, returning (subnet, gateway, bundle_ip). bundle_ip is always at .2 (gateway .1); subnet is /24, third octet derived from the slug hash, skipping the docker-default 17 to avoid the common 192.168.17.x collision. - allocate_loopback_port: deleted. The bundle gets a pinned docker IP now; the agent dials that IP directly through TSI. - smolmachines_preflight: dropped the gvproxy check; only smolvm is required. prepare.py: - Drops the gvproxy.yaml render + the loopback port allocation + the gvproxy_socket field on the plan. - Derives subnet / gateway / bundle_ip from the slug and populates the new SmolmachinesBottlePlan fields. - Agent env now uses IP-literal URLs (http://<bundle-ip>:8888 etc) since the guest will have no DNS resolver inside TSI's allowlist. bottle_plan.py: - Old fields: gvproxy_config_path, gvproxy_socket, gvproxy_subnet, gvproxy_gateway, host_port_map. - New fields: bundle_subnet, bundle_gateway, bundle_ip, smolfile_path. (smolmachine artifact path lands in chunk 2b.) Net: -410 lines. Full unit suite: 516 passing. The VM lifecycle + bundle bringup + launch wiring + smoke tests land in chunk 2b. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
 didericis
|
20f411b22e |
feat(smolmachines): backend skeleton + Smolfile/gvproxy renderers (PRD 0023 chunk 1)
Ships the smolmachines backend's prepare side: subpackage layout,
`_BACKENDS` registration under "smolmachines", preflight check
for `smolvm` + `gvproxy` on PATH, and the two config-file
renderers (Smolfile TOML + gvproxy YAML). Launch raises
NotImplementedError until chunk 2.
New module layout (mirrors backend/docker/):
claude_bottle/backend/smolmachines/
__init__.py re-exports SmolmachinesBottleBackend
backend.py SmolmachinesBottleBackend façade
bottle.py SmolmachinesBottle stub (NotImpl until ch2)
bottle_plan.py SmolmachinesBottlePlan + .print()
bottle_cleanup_plan.py SmolmachinesBottleCleanupPlan stub
prepare.py resolve_plan: writes both config files
smolfile.py TOML renderer (stdlib, no tomli_w dep)
gvproxy_config.py YAML renderer (same shape as pipelock_yaml)
util.py preflight + per-slug subnet + loopback port
The renderers are pure functions. `resolve_plan` runs the
preflight, allocates one host-side loopback port per active
sidecar (pipelock always; git-gate / supervise conditional),
derives a per-slug gvproxy subnet (hash-mod-254, skipping the
docker-default 17), and writes:
- <stage>/gvproxy.yaml: subnet + DNS rule resolving only
`proxy.internal` + port_forwards (one per active sidecar).
- <stage>/smolfile.toml: guest command/env + virtio-net device
backed by gvproxy's unixgram socket. No TSI flags — see
PRD 0023 "Why gvproxy, not TSI".
The agent's HTTPS_PROXY etc. point at `proxy.internal:<gateway-
port>` so the guest dials through gvproxy. gvproxy resolves only
`proxy.internal` → the gateway IP, and forwards exactly the
listed ports to the host-side sidecar bundle (PRD 0024); every
other destination — host LAN, host loopback, public internet
directly — is unreachable by construction.
29 new unit tests covering renderer correctness, subnet
derivation stability + collision-avoidance, loopback port
allocation, and preflight error paths. Full unit suite: 532
passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|