feat(macos): consolidated per-host gateway for the Apple backend (PRD 0070)
Re-enables the macos-container backend on the shared per-host orchestrator + gateway, replacing the per-bottle companion container removed in #385. This is the last backend in PRD 0070's roadmap. Apple Container 1.0.0 forced three departures from the docker shape, each verified against the live CLI (findings recorded in the networking spike): - No `--ip`. The address is DHCP-assigned and knowable only once the container runs, so the order inverts: gateway up -> run agent -> read its address -> register. The identity token is minted by registration and therefore cannot be in the agent's run-time env; it rides the proxy URL applied at `container exec` time (bare `--env` names keep it off argv). - No container DNS. The gateway can only be handed the control plane's IP, so the orchestrator starts first and the gateway is pointed at its address. - No `network connect`. Networks are fixed at run time, so the shared host-only network is created up front; per-bottle networks would restart the gateway on every launch and defeat the consolidation. The agent runs with `--cap-drop CAP_NET_RAW`: Apple grants NET_RAW by default, which would let an agent forge a neighbour's source address on the shared segment. NET_ADMIN is already absent, so this closes the source-address half of PRD 0070's attribution invariant. Verified end-to-end on real Apple Container 1.0.0: both images build, the control plane comes up healthy, the gateway reaches it by IP, and a registered agent gets 200 for a host in its routes and 403 for one outside them. Bring-up is idempotent — a second launch does not churn the singletons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -358,3 +358,111 @@ the Apple Container-specific constraints directly:
|
||||
|
||||
Do not implement the backend as a direct clone of Docker Compose
|
||||
service aliases. That assumption failed in this run.
|
||||
|
||||
## Addendum: consolidated-gateway findings (2026-07-17, PRD 0070)
|
||||
|
||||
Re-tested on Apple Container 1.0.0 while porting the backend to the
|
||||
per-host consolidated gateway (#351). The two-network shape above still
|
||||
holds; these are the additional constraints that shaped the port, each
|
||||
verified against the live CLI on this host.
|
||||
|
||||
### No static IP for a container
|
||||
|
||||
`container run --network` accepts only
|
||||
`<name>[,mac=XX:XX:XX:XX:XX:XX][,mtu=VALUE]`. There is no `--ip`. The
|
||||
address comes from vmnet's DHCP and is knowable only after the container
|
||||
is running:
|
||||
|
||||
```console
|
||||
$ container run --name a --network bb-net --detach alpine sleep 900
|
||||
$ container inspect a | jq -r '.[0].status.networks[0].ipv4Address'
|
||||
192.168.128.3/24
|
||||
```
|
||||
|
||||
Consequence: the docker backend's "allocate a free IP -> pin it with
|
||||
`--ip` -> register -> launch" order cannot be reproduced. macOS inverts
|
||||
it to "launch -> read the assigned address -> register". The identity
|
||||
token therefore cannot be in the agent's run-time env (registration mints
|
||||
it after the container exists) and is delivered at `container exec` time.
|
||||
|
||||
### Networks are fixed at run time
|
||||
|
||||
There is no `container network connect`; `container network` exposes only
|
||||
`create`, `delete`, `list`, `inspect`, `prune`. A network cannot be
|
||||
attached to a running container, so a *persistent* shared gateway rules
|
||||
out per-bottle networks — they would force a gateway restart per launch.
|
||||
One shared host-only network, created up front, is the only shape that
|
||||
keeps the gateway a singleton.
|
||||
|
||||
### No container DNS
|
||||
|
||||
Containers cannot resolve each other by name; the host-only network's
|
||||
resolver refuses the query:
|
||||
|
||||
```console
|
||||
$ container exec agent nslookup gw
|
||||
;; connection timed out; no servers could be reached
|
||||
$ container exec agent cat /etc/resolv.conf
|
||||
nameserver 192.168.128.1
|
||||
```
|
||||
|
||||
Consequence: the gateway is handed the control plane's **IP**, not a
|
||||
container name as on docker. That forces the startup order
|
||||
orchestrator -> read its address -> gateway.
|
||||
|
||||
### The host can reach the host-only network directly
|
||||
|
||||
```console
|
||||
$ container inspect c | jq -r '.[0].status.networks[0].ipv4Address'
|
||||
192.168.128.2/24
|
||||
$ curl -s http://192.168.128.2:8099/i
|
||||
ok
|
||||
```
|
||||
|
||||
So no `--publish` hop is needed: the host CLI and the gateway use the
|
||||
same control-plane URL. Docker needs `--publish 127.0.0.1:...` plus a
|
||||
separate internal URL for the same job.
|
||||
|
||||
### CAP_NET_RAW is granted by default — and matters for attribution
|
||||
|
||||
Apple grants NET_RAW but not NET_ADMIN. The agent therefore cannot change
|
||||
its own address or route:
|
||||
|
||||
```console
|
||||
$ container exec agent ip addr add 192.168.128.99/24 dev eth0
|
||||
ip: RTNETLINK answers: Operation not permitted
|
||||
$ container exec agent ip route replace default via 192.168.128.2 dev eth0
|
||||
ip: RTNETLINK answers: Operation not permitted
|
||||
$ container exec agent grep CapEff /proc/self/status
|
||||
CapEff: 00000000a80425fb # bit 13 (NET_RAW) set, bit 12 (NET_ADMIN) clear
|
||||
```
|
||||
|
||||
But NET_RAW permits raw sockets, i.e. source-address forgery against
|
||||
neighbours on the shared segment — directly against PRD 0070's invariant
|
||||
("a packet's source address, as seen by the orchestrator, provably
|
||||
identifies the originating bottle"). `--cap-drop CAP_NET_RAW` closes it:
|
||||
|
||||
```console
|
||||
$ container run --cap-drop CAP_NET_RAW ... alpine
|
||||
$ container exec nr grep CapEff /proc/self/status
|
||||
CapEff: 00000000a80405fb # bit 13 cleared
|
||||
$ container exec nr ping -c1 192.168.128.2
|
||||
ping: permission denied (are you root?)
|
||||
```
|
||||
|
||||
The agent is run with `--cap-drop CAP_NET_RAW` for this reason.
|
||||
|
||||
### `container exec` inherits run-time env, and `--env` overrides it
|
||||
|
||||
```console
|
||||
$ container run --name e --env FOO=from_run --detach alpine sleep 120
|
||||
$ container exec e sh -c 'echo $FOO'
|
||||
from_run
|
||||
$ container exec --env FOO=from_exec e sh -c 'echo $FOO'
|
||||
from_exec
|
||||
```
|
||||
|
||||
This is what makes exec-time identity-token delivery work: the token-less
|
||||
proxy URL baked in at launch is superseded by the token-bearing one at
|
||||
exec. Bare `--env NAME` (inherit from the parent process) keeps the token
|
||||
value off argv.
|
||||
|
||||
Reference in New Issue
Block a user