docs(prd-0070): resolve build-placement open question — builds stay in the orchestrator
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 31s
test / unit (pull_request) Successful in 46s
test / integration-firecracker (pull_request) Successful in 3m23s
test / coverage (pull_request) Successful in 42s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 31s
test / unit (pull_request) Successful in 46s
test / integration-firecracker (pull_request) Successful in 3m23s
test / coverage (pull_request) Successful in 42s
test / publish-infra (pull_request) Has been skipped
v1 decision: agent-image builds run in the orchestrator (the control-plane side already owns launches and hosts builds in today's combined unit; the data-plane gateway must not build). Consequence recorded in the memory section: the orchestrator keeps the ~4 GB buildah ceiling and the gateway is the slim unit (~1 GB), so the split is ~memory-neutral-to-+1 GB for now — the ~1.5 GB win waits on a later move to a dedicated slim build unit (#468-adjacent). Also flags the tension of co-locating buildah with the control plane. Removed the corresponding open question. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -427,27 +427,32 @@ unchanged. A *compromised gateway* can already reach the orchestrator today
|
||||
the split adds **no new surface** for the gateway-compromise case; it only
|
||||
closes the *agent* path on Docker/macOS.
|
||||
|
||||
#### Memory: right-size, don't double
|
||||
#### Memory: fixed ceilings, and where builds run
|
||||
|
||||
Firecracker guest memory is a **fixed ceiling** set at boot (`mem_size_mib`);
|
||||
it's demand-paged by the host (RSS grows with what the guest touches) but there
|
||||
is **no reclaim without a balloon device**, and none is configured — so plan
|
||||
for the ceiling. The combined infra VM is **4096 MiB**, but that headroom is
|
||||
for **buildah** (in-guest agent-image builds), not the daemons, which are
|
||||
light.
|
||||
light. Steady-state daemon footprints are small: the **orchestrator**
|
||||
(stdlib-only Python HTTP + SQLite) is a few hundred MiB; the **gateway**
|
||||
(mitmproxy TLS bump + DLP body buffering, git-http, supervise, gitleaks) is
|
||||
**512 MiB–1 GB**.
|
||||
|
||||
Split naïvely that reads as 2×, but right-sized it is **less** than today:
|
||||
|
||||
- **Orchestrator** — stdlib-only Python HTTP + SQLite, no builds → **256–512
|
||||
MiB**.
|
||||
- **Gateway** — mitmproxy (TLS bump + DLP body buffering) + git-http +
|
||||
supervise + gitleaks → **512 MiB–1 GB**, *if it does not build images*.
|
||||
|
||||
The real lever is **where agent-image builds run** (buildah's 2–4 GB working
|
||||
set). Move builds off the gateway — to a host-controller / dedicated build unit
|
||||
(a #468-adjacent decision) — and a slim orchestrator (~512 MiB) + slim gateway
|
||||
(~1 GB) ≈ **1.5 GB total vs. today's 4 GB**. A per-unit **balloon device** to
|
||||
reclaim idle memory is a later option, not needed for v1.
|
||||
The real ceiling driver is **where agent-image builds run** (buildah's 2–4 GB
|
||||
working set). **v1 decision: builds stay in the orchestrator** — it is the
|
||||
control-plane/management side that owns launches (and already hosts builds in
|
||||
today's combined unit), and the data-plane gateway must not build. So the
|
||||
**orchestrator keeps a ~4 GB ceiling** (mostly idle, spiking during a build)
|
||||
and the **gateway is the slim unit** (~1 GB). Net for v1 the split is therefore
|
||||
roughly **memory-neutral to ~+1 GB** — a slim gateway added beside a
|
||||
still-build-capable orchestrator — *not* the ~1.5 GB total; that win is
|
||||
deferred to a later move of builds onto a **dedicated slim build unit**
|
||||
(#468-adjacent). Co-locating buildah with the control plane also keeps a
|
||||
network-heavy, image-defined workload next to the signing key + registry — a
|
||||
known tension the dedicated build unit resolves later. A per-unit **balloon
|
||||
device** to reclaim the orchestrator's build-vs-idle swing is the other future
|
||||
lever. The access and surface-separation gains land now regardless.
|
||||
|
||||
#### iroh belongs in the orchestrator
|
||||
|
||||
@@ -550,11 +555,6 @@ Keep the gateway **service one shared thing** throughout.
|
||||
plane (add/remove routes/keys/proposals without a restart).
|
||||
- **Identity-token delivery:** exactly how the per-bottle token is placed
|
||||
where the agent can present it but not swap in another bottle's.
|
||||
- **Where agent-image builds run once the planes split:** buildah's 2–4 GB
|
||||
working set is what keeps the combined infra VM at 4 GB. Keeping the gateway
|
||||
slim (~1 GB) means builds move off it — to a host-controller / dedicated
|
||||
build unit (#468-adjacent) — or the gateway keeps the build headroom and the
|
||||
memory win is smaller. Decides the gateway VM/container size.
|
||||
- **Agent-PTY-to-mobile streaming path:** relay it through the orchestrator's
|
||||
iroh endpoint (default — one authenticated remote door, plane split intact),
|
||||
or give the data-plane stream its own transport. Only matters once remote
|
||||
|
||||
Reference in New Issue
Block a user