feat(smolmachines): thread inner Plans + bundle daemons run (PRD 0023 chunk 4b) #70

Merged
didericis-claude merged 5 commits from prd-0023-chunk-4b-inner-plans into main 2026-05-27 13:21:42 -04:00
Collaborator

Summary

Chunk 4b: bundle daemons now actually start with their config files. Chunks 2d + 3 ran with daemons_csv="" so the bundle's init supervisor idled; chunk 4b wires up the real path. From inside the smolvm guest, wget http://<bundle-ip>:8888/ now reaches a live pipelock.

The remaining gap to full end-to-end is the agent-image-conversion (chunk 4c) — the chunk-2d alpine placeholder doesn't have claude-code, git, curl, or a writable trust store, so provision_ca / provision_git / provision_supervise still wait (chunk 4d).

Mechanism

  • bottle_plan.py — new fields proxy_plan / git_gate_plan / egress_plan / supervise_plan. Same shape the docker backend's plan uses; docker-network-only fields stay at dataclass defaults.
  • prepare.py — instantiates DockerPipelockProxy / DockerEgress / DockerGitGate / DockerSupervise and calls their .prepare() methods to write per-bottle config files (pipelock.yaml, routes.yaml, git-gate entrypoint/hooks, supervise queue dir) under per-bottle state dirs. The .prepare() methods live on the platform-neutral ABCs — the "Docker" class names are misleading but it's a one-line cleanup for a future PR.
  • launch.py — pipelock_tls_init at launch (always); egress_tls_init only when routes declared. Inner Plans updated in place with the CA paths + EGRESS_UPSTREAM_PROXY=http://127.0.0.1:8888 (egress's upstream is pipelock on the bundle's own loopback; same container's network namespace). BundleLaunchSpec env + volumes built from the inner Plans. daemons_csv = ["egress", "pipelock"] + ["git-gate"] (if upstreams) + ["supervise"] (if enabled). Token env values resolved from host env and threaded through the docker-run subprocess env.

Tests

  • 552 unit passing (no new cases; fixture updated to populate the new plan fields).
  • 5 integration cases passing locally (Darwin + smolvm + docker, not GITEA_ACTIONS):
    • test_smoke_exec_echo — still works.
    • test_localhost_reach_probe — host loopback still refused.
    • test_egress_port_bypass_probe<bundle-ip>:9099 still refused, NOW WITH EGRESS ACTUALLY RUNNING. Chunk 3's 127.0.0.1 bind-address is doing its job.
    • test_prompt_file_lands_in_guest — still works.
    • test_pipelock_answers_on_bundle_ip — NEW. wget to <bundle-ip>:8888 gets an HTTP response (not "connection refused"). Proves pipelock is listening with its config + CA bind-mounts intact.

Roadmap for the rest of chunk 4

  • 4c — agent-image-conversion: get claude-code + git + curl + ca-certificates into the guest. Two paths: smolvm pack create --from-vm after manual VM setup, or push the docker image to a smolvm-reachable registry.
  • 4dprovision_ca + provision_git + provision_supervise once 4c lands.
## Summary Chunk 4b: bundle daemons now actually start with their config files. Chunks 2d + 3 ran with `daemons_csv=""` so the bundle's init supervisor idled; chunk 4b wires up the real path. From inside the smolvm guest, `wget http://<bundle-ip>:8888/` now reaches a live pipelock. The remaining gap to full end-to-end is the agent-image-conversion (chunk 4c) — the chunk-2d alpine placeholder doesn't have claude-code, git, curl, or a writable trust store, so `provision_ca` / `provision_git` / `provision_supervise` still wait (chunk 4d). ## Mechanism - **`bottle_plan.py`** — new fields `proxy_plan` / `git_gate_plan` / `egress_plan` / `supervise_plan`. Same shape the docker backend's plan uses; docker-network-only fields stay at dataclass defaults. - **`prepare.py`** — instantiates `DockerPipelockProxy` / `DockerEgress` / `DockerGitGate` / `DockerSupervise` and calls their `.prepare()` methods to write per-bottle config files (pipelock.yaml, routes.yaml, git-gate entrypoint/hooks, supervise queue dir) under per-bottle state dirs. The `.prepare()` methods live on the platform-neutral ABCs — the "Docker" class names are misleading but it's a one-line cleanup for a future PR. - **`launch.py`** — pipelock_tls_init at launch (always); egress_tls_init only when routes declared. Inner Plans updated in place with the CA paths + EGRESS_UPSTREAM_PROXY=`http://127.0.0.1:8888` (egress's upstream is pipelock on the bundle's own loopback; same container's network namespace). `BundleLaunchSpec` env + volumes built from the inner Plans. `daemons_csv` = `["egress", "pipelock"]` + `["git-gate"]` (if upstreams) + `["supervise"]` (if enabled). Token env values resolved from host env and threaded through the docker-run subprocess env. ## Tests - **552 unit passing** (no new cases; fixture updated to populate the new plan fields). - **5 integration cases passing locally** (Darwin + smolvm + docker, not `GITEA_ACTIONS`): * `test_smoke_exec_echo` — still works. * `test_localhost_reach_probe` — host loopback still refused. * `test_egress_port_bypass_probe` — `<bundle-ip>:9099` still refused, **NOW WITH EGRESS ACTUALLY RUNNING**. Chunk 3's `127.0.0.1` bind-address is doing its job. * `test_prompt_file_lands_in_guest` — still works. * `test_pipelock_answers_on_bundle_ip` — NEW. `wget` to `<bundle-ip>:8888` gets an HTTP response (not "connection refused"). Proves pipelock is listening with its config + CA bind-mounts intact. ## Roadmap for the rest of chunk 4 - **4c** — agent-image-conversion: get claude-code + git + curl + ca-certificates into the guest. Two paths: `smolvm pack create --from-vm` after manual VM setup, or push the docker image to a smolvm-reachable registry. - **4d** — `provision_ca` + `provision_git` + `provision_supervise` once 4c lands.
didericis-claude added 1 commit 2026-05-27 05:29:24 -04:00
feat(smolmachines): thread inner Plans + bundle daemons run (PRD 0023 chunk 4b)
test / unit (pull_request) Successful in 21s
test / integration (pull_request) Successful in 42s
1dfc359141
Bundle daemons (pipelock, egress, optionally git-gate + supervise)
now actually start with their config files bind-mounted from the
inner Plans the docker backend already produces. Chunks 2d + 3
ran with daemons_csv="" so the bundle's init supervisor idled;
chunk 4b wires up the real path: agent → pipelock → egress →
internet (when routes declared) is now functional, modulo agent-
image gaps (claude-code / TLS-trust-store / git in the guest)
that chunk 4c addresses.

bottle_plan.py — added the four inner Plan fields:
  proxy_plan: PipelockProxyPlan
  git_gate_plan: GitGatePlan
  egress_plan: EgressPlan
  supervise_plan: SupervisePlan | None

Same shape the docker backend's plan uses. Docker-network-only
fields (internal_network, egress_network) stay at dataclass
defaults — the smolmachines bundle is on a per-bottle bridge
with a pinned IP, not docker's --internal + egress topology.

prepare.py — instantiates DockerPipelockProxy / DockerEgress /
DockerGitGate / DockerSupervise and calls their .prepare()
methods to write the per-bottle config files (pipelock.yaml,
routes.yaml, git-gate entrypoint/hooks, supervise queue dir)
under the per-bottle state dir. (The "Docker" prefix on the
class names is a misnomer here — .prepare() is platform-neutral,
inherited from each sidecar's ABC. A future cleanup could factor
the prepare logic out of the docker subpackage.)

launch.py — major rewrite:
  - pipelock_tls_init at launch (always); egress_tls_init only
    when the bottle declares routes (otherwise the CA files
    aren't bind-mounted and openssl runs would be wasted).
  - Inner Plans updated in place with launch-time CA paths +
    EGRESS_UPSTREAM_PROXY = http://127.0.0.1:8888 (egress's
    upstream is pipelock on the bundle's own loopback; same
    container's network namespace).
  - BundleLaunchSpec env + volumes built from the inner Plans:
    pipelock.yaml + CA + key (always); egress routes + CAs +
    upstream env + token-slot bare names (when routes); git-gate
    entrypoint + hooks + per-upstream identity files (when
    upstreams); supervise queue dir + env (when enabled).
  - daemons_csv = ["egress", "pipelock"] + ["git-gate"] (if
    upstreams) + ["supervise"] (if enabled).
  - Token env values resolved from host env via
    `egress_resolve_token_values` and threaded into the
    docker-run subprocess env (bare-name -e entries in spec
    inherit from there — values never land on argv).

Tests:
- 552 unit passing (no new unit cases; fixture updated to
  populate the new plan fields).
- 5 integration cases passing locally (Darwin + smolvm + docker
  + not GITEA_ACTIONS):
    * test_smoke_exec_echo — still works.
    * test_localhost_reach_probe — host loopback still refused.
    * test_egress_port_bypass_probe — <bundle-ip>:9099 still
      refused, NOW WITH EGRESS ACTUALLY RUNNING (chunk 3's
      127.0.0.1 bind-address is doing its job).
    * test_prompt_file_lands_in_guest — still works.
    * test_pipelock_answers_on_bundle_ip — NEW. From inside the
      guest, wget to <bundle-ip>:8888 gets an HTTP response
      (not "connection refused") — proves pipelock is actually
      listening and the bind-mount + CA generation path works.

What's left in chunk 4:
- 4c: agent-image-conversion (claude-code + git + curl +
  ca-certificates in the guest). Chunk 2d's alpine placeholder
  stays for now.
- 4d: provision_ca + provision_git + provision_supervise once
  the agent image has the required tools.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis requested changes 2026-05-27 05:32:33 -04:00
Dismissed
@@ -89,0 +98,4 @@
# backend's concrete subclasses — the `.prepare()` method
# they inherit is platform-neutral (writes config files +
# returns a Plan dataclass); the docker-specific subclasses
# exist only to satisfy ABC instantiation. Future: factor
Owner

Do this refactor now

Do this refactor now
didericis marked this conversation as resolved
didericis added 1 commit 2026-05-27 05:42:24 -04:00
refactor(sidecars): instantiate sidecar ABCs directly from any backend
test / unit (pull_request) Successful in 21s
test / integration (pull_request) Successful in 40s
73dc0d4a40
The four sidecar prepare-time helpers (PipelockProxy, Egress, GitGate,
Supervise) had docker-flavored subclasses that existed only as
instantiation shims for ABCs that already had no abstract methods.
PipelockProxy.prepare() reached for class-level CA path constants
that were only defined on the docker subclass — so smolmachines had
to import DockerPipelockProxy to render pipelock yaml, reaching
across the backend boundary for what's actually a platform-neutral
operation.

This moves the universal in-container CA paths
(PIPELOCK_CA_CERT_IN_CONTAINER / PIPELOCK_CA_KEY_IN_CONTAINER) to
claude_bottle/pipelock.py, drops the class-attr indirection on the
ABC, and deletes the four empty docker subclasses. Both backends
now instantiate the ABCs directly; the docker-side modules keep
the docker-flavored helpers (image pin, container naming, host CA
mint) and re-export the moved pipelock constants for compat.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis-claude reviewed 2026-05-27 12:45:13 -04:00
@@ -89,0 +98,4 @@
# platform-neutral — `.prepare()` writes config files + returns
# a Plan dataclass with no backend-specific assumptions. State
# dirs are still keyed by slug under the docker backend's
# bottle_state layout (shared on-host convention; not a docker
Author
Collaborator

Done in 73dc0d4 — moved CA in-container path constants up to claude_bottle/pipelock.py, made PipelockProxy a regular class, and deleted the four empty Docker* sidecar subclasses. Both backends now instantiate the platform-neutral ABCs directly.

Done in 73dc0d4 — moved CA in-container path constants up to `claude_bottle/pipelock.py`, made `PipelockProxy` a regular class, and deleted the four empty `Docker*` sidecar subclasses. Both backends now instantiate the platform-neutral ABCs directly.
didericis approved these changes 2026-05-27 12:47:21 -04:00
Dismissed
didericis reviewed 2026-05-27 12:49:00 -04:00
@@ -5,1 +1,3 @@
the container-name helper the renderer's network alias targets."""
"""Docker-side supervise helpers: container naming for the legacy
per-sidecar topology (kept so the bundle's docker-network alias
resolves the old name to the bundle IP). The prepare-time
Owner

We don't care about preserving the legacy topology

We don't care about preserving the legacy topology
didericis added 1 commit 2026-05-27 12:52:54 -04:00
refactor(docker): drop legacy supervise_container_name alias
test / unit (pull_request) Successful in 22s
test / integration (pull_request) Successful in 40s
8ecba2b458
Supervise runs inside the sidecar bundle (PRD 0024), not in its own
container. The `claude-bottle-supervise-<slug>` per-sidecar name only
existed as a docker-network alias on the bundle so legacy code paths
that referenced the old name would still resolve. Nothing inside the
project relies on that resolution anymore — the short `supervise`
alias is the one all consumers use — so the legacy long-form is dead.

Drops the function entirely, plus its registration as a network alias
and as an orphan probe in prepare.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis-claude reviewed 2026-05-27 12:53:02 -04:00
@@ -1,3 +1,3 @@
"""Docker-side supervise helpers: container naming for the legacy
per-sidecar topology (kept so the bundle's docker-network alias
resolves the old name to the bundle IP). The prepare-time
Author
Collaborator

Dropped in 8ecba2b — deleted claude_bottle/backend/docker/supervise.py entirely (only legacy-name function left after the refactor), removed the network alias in compose.py, removed the orphan probe in prepare.py, and stopped asserting the long-form alias in test_compose.py. The bundle keeps the short supervise alias since SUPERVISE_HOSTNAME is what consumers actually use.

Dropped in 8ecba2b — deleted `claude_bottle/backend/docker/supervise.py` entirely (only legacy-name function left after the refactor), removed the network alias in `compose.py`, removed the orphan probe in `prepare.py`, and stopped asserting the long-form alias in `test_compose.py`. The bundle keeps the short `supervise` alias since `SUPERVISE_HOSTNAME` is what consumers actually use.
Owner

The other container name functions can probably go too, right?

The other container name functions can probably go too, right?
didericis added 1 commit 2026-05-27 13:04:53 -04:00
refactor(docker): drop legacy per-sidecar container_name functions
test / unit (pull_request) Successful in 21s
test / integration (pull_request) Successful in 41s
727f30d422
Same line of cleanup as the supervise rename: the per-sidecar
container names (`claude-bottle-pipelock-<slug>`,
`claude-bottle-egress-<slug>`, `claude-bottle-git-gate-<slug>`)
were docker-network aliases pointing at the bundle, kept so legacy
URLs would keep resolving. Replaces them with short hostnames
(`pipelock`, `egress`, `git-gate`) matching the existing
`EGRESS_HOSTNAME` pattern, and inlines the bundle-loopback URL
(`http://127.0.0.1:8888`) for the in-bundle egress→pipelock hop —
matching what smolmachines already does.

Drops the three `*_container_name` functions, `pipelock_proxy_url`,
and `git_gate_host`. Their callers move to the new constants:
- `PIPELOCK_HOSTNAME = "pipelock"` (claude_bottle/pipelock.py)
- `GIT_GATE_HOSTNAME = "git-gate"` (claude_bottle/git_gate.py)
- `BUNDLE_LOCAL_PIPELOCK_URL` (backend/docker/pipelock.py)

The agent's HTTP_PROXY now reads `http://pipelock:8888` (vs the
old `http://claude-bottle-pipelock-<slug>:8888`); the gitconfig
insteadOf rewrites become `git://git-gate/<repo>.git`. The prepare-
time orphan probe is collapsed onto the bundle container name
(`claude-bottle-sidecars-<slug>`) instead of the four legacy
per-sidecar names that no backend creates anymore.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis-claude reviewed 2026-05-27 13:05:08 -04:00
@@ -1,3 +1,3 @@
"""Docker-side supervise helpers: container naming for the legacy
per-sidecar topology (kept so the bundle's docker-network alias
resolves the old name to the bundle IP). The prepare-time
Author
Collaborator

Yep — done in 727f30d. Killed pipelock_container_name, egress_container_name, git_gate_container_name, plus pipelock_proxy_url and git_gate_host since they only existed to build URLs with the legacy long-form names. Added PIPELOCK_HOSTNAME = "pipelock" and GIT_GATE_HOSTNAME = "git-gate" (mirroring EGRESS_HOSTNAME = "egress"), so the agent's HTTPS_PROXY is now http://pipelock:8888 and the gitconfig insteadOf rewrites are git://git-gate/<repo>.git. The bundle-internal egress→pipelock hop drops the DNS alias entirely and uses loopback (BUNDLE_LOCAL_PIPELOCK_URL = http://127.0.0.1:8888) — same pattern smolmachines was already using.

The prepare-time orphan probe collapses to one check on the bundle container name (claude-bottle-sidecars-<slug>) instead of the four legacy per-sidecar names nothing creates anymore.

One thing I left alone: capability_apply._per_bottle_container_names still lists the four legacy names in its teardown helper. Harmless (missing containers are silently skipped) but technically also dead. Easy to wipe in a follow-up if you want.

Yep — done in 727f30d. Killed `pipelock_container_name`, `egress_container_name`, `git_gate_container_name`, plus `pipelock_proxy_url` and `git_gate_host` since they only existed to build URLs with the legacy long-form names. Added `PIPELOCK_HOSTNAME = "pipelock"` and `GIT_GATE_HOSTNAME = "git-gate"` (mirroring `EGRESS_HOSTNAME = "egress"`), so the agent's HTTPS_PROXY is now `http://pipelock:8888` and the gitconfig insteadOf rewrites are `git://git-gate/<repo>.git`. The bundle-internal egress→pipelock hop drops the DNS alias entirely and uses loopback (`BUNDLE_LOCAL_PIPELOCK_URL = http://127.0.0.1:8888`) — same pattern smolmachines was already using. The prepare-time orphan probe collapses to one check on the bundle container name (`claude-bottle-sidecars-<slug>`) instead of the four legacy per-sidecar names nothing creates anymore. One thing I left alone: `capability_apply._per_bottle_container_names` still lists the four legacy names in its teardown helper. Harmless (missing containers are silently skipped) but technically also dead. Easy to wipe in a follow-up if you want.
Owner

wipe that out too

wipe that out too
didericis added 1 commit 2026-05-27 13:07:18 -04:00
refactor(docker): drop legacy names from capability_apply teardown
test / unit (pull_request) Successful in 21s
test / integration (pull_request) Successful in 40s
519a71f2e7
Last of the per-sidecar legacy names. `_per_bottle_container_names`
used to list the four pre-bundle sidecars (cred-proxy, pipelock,
git-gate, supervise) so capability-apply's teardown would force-rm
them on remediation. None of those containers exist anymore — the
four daemons run in the sidecar bundle (PRD 0024), so the list
collapses to the agent + the bundle.

Integration test follows: the fake supervise-sidecar setup, which
existed to give teardown an extra container to clean up, switches
to a fake sidecar bundle with the current name.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
didericis-claude reviewed 2026-05-27 13:07:26 -04:00
@@ -1,3 +1,3 @@
"""Docker-side supervise helpers: container naming for the legacy
per-sidecar topology (kept so the bundle's docker-network alias
resolves the old name to the bundle IP). The prepare-time
Author
Collaborator

Done in 519a71f. _per_bottle_container_names is now just [agent, sidecar-bundle]; the four legacy per-sidecar literals are gone. Integration test follows — the fake supervise sidecar (which existed to give teardown an extra container to nuke) switches to a fake sidecar bundle.

Done in 519a71f. `_per_bottle_container_names` is now just `[agent, sidecar-bundle]`; the four legacy per-sidecar literals are gone. Integration test follows — the fake supervise sidecar (which existed to give teardown an extra container to nuke) switches to a fake sidecar bundle.
didericis approved these changes 2026-05-27 13:20:43 -04:00
didericis-claude merged commit 4ac61a563b into main 2026-05-27 13:21:42 -04:00
didericis deleted branch prd-0023-chunk-4b-inner-plans 2026-05-27 14:02:15 -04:00
Sign in to join this conversation.