2f8539c2c7
test / integration-docker (pull_request) Successful in 14s
test / unit (pull_request) Successful in 38s
tracker-policy-pr / check-pr (pull_request) Successful in 25s
lint / lint (push) Successful in 49s
test / stage-firecracker-inputs (pull_request) Successful in 5s
test / build-infra (pull_request) Successful in 3m31s
test / integration-firecracker (pull_request) Successful in 1m51s
test / coverage (pull_request) Failing after 1m33s
test / publish-infra (pull_request) Has been skipped
The shared gateway's address is DHCP-assigned and changes whenever the infra container is recreated — a source-hash bump, an image upgrade, a crash. Every agent-facing URL embedded that address, and the proxy URL reaches the agent as process environment at `container exec` time. A running process's environ cannot be rewritten from outside, so a moved gateway stranded every running bottle permanently: not degraded, unreachable, until relaunched and its agent session thrown away. Give the agent a stable name instead. `GATEWAY_HOSTNAME` replaces the address in the egress proxy URL, NO_PROXY, git-http, and supervise URLs, and resolves through the bottle's own /etc/hosts. Unlike environ that is a file, so it can be rewritten inside a container that is already running — which is the whole point: a gateway that returns at a new address is picked up by live bottles. Launch writes the entry before anything execs (every agent URL names the gateway, so it must resolve for the first connection), and re-points every running bottle once the gateway is up, so one stranded by an earlier restart re-attaches instead of needing a relaunch. The write needs root and the agent runs as `node`: the host can repoint a bottle's gateway name, the agent cannot repoint its own. Keep that asymmetry. Apple Container 1.0 has no container-name DNS on a user network and `container run` has no --add-host, so the entry is written by exec after start rather than declared at run. Does not address the other half of #443: per-bottle egress auth tokens are held in memory by the orchestrator and are still lost across a restart, so a re-attached bottle resolves its policy but not its injected credentials. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
97 lines
4.1 KiB
Python
97 lines
4.1 KiB
Python
"""Stable gateway name for macOS agents, via each bottle's `/etc/hosts`.
|
|
|
|
The shared gateway's address is assigned by vmnet's DHCP and changes whenever
|
|
the infra container is recreated — a source-hash bump, an image upgrade, a
|
|
crash. Every agent-facing URL (egress proxy, git-http, supervise) embeds that
|
|
address, and the proxy URL reaches the agent as **process environment** at
|
|
`container exec` time. A running process's `environ` cannot be rewritten from
|
|
outside, so a moved gateway used to strand every running bottle permanently:
|
|
not degraded, unreachable, until the bottle was relaunched and its session
|
|
thrown away.
|
|
|
|
So the agent never learns the address. It is given a stable *name*
|
|
(`GATEWAY_HOSTNAME`) in every URL, resolved through its own `/etc/hosts`.
|
|
Unlike `environ`, that is a file — it can be rewritten inside a container that
|
|
is already running, so a gateway that comes back at a new address is picked up
|
|
by live bottles instead of orphaning them.
|
|
|
|
Apple Container 1.0 offers no container-name DNS on a user network (the only
|
|
nameserver an agent sees is vmnet's, which does not know container names) and
|
|
`container run` has no `--add-host`, so the entry is written by exec after the
|
|
container starts.
|
|
|
|
Writing it needs root, and the agent runs as `node`: the agent therefore
|
|
cannot repoint its own gateway name, while the host (which drives `container
|
|
exec --user root`) can. That asymmetry is deliberate — keep it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from ...log import warn
|
|
from . import util as container_mod
|
|
from .enumerate import CONTAINER_NAME_PREFIX, enumerate_active
|
|
|
|
# The name every agent-facing gateway URL uses. Must not collide with a real
|
|
# DNS name the agent might resolve; it is bottle-local by construction.
|
|
GATEWAY_HOSTNAME = "bot-bottle-gateway"
|
|
|
|
# Marker so the rewrite is idempotent and only ever touches our own line —
|
|
# the rest of /etc/hosts (localhost, the container's own name) is preserved.
|
|
_MARKER = "# bot-bottle gateway"
|
|
|
|
|
|
def _rewrite_script(gateway_ip: str) -> str:
|
|
"""A shell one-liner that replaces our managed line in `/etc/hosts`.
|
|
|
|
Rewrites in place via a temp file + `cat` rather than `mv`, so the file
|
|
keeps its original inode, ownership, and mode — a bind-mounted or
|
|
pre-created `/etc/hosts` must not be replaced by a root-owned 0644 copy
|
|
that the runtime then refuses to update.
|
|
"""
|
|
return (
|
|
"set -e; "
|
|
f"grep -v '{_MARKER}' /etc/hosts > /tmp/.bb-hosts || true; "
|
|
f"printf '%s %s %s\\n' '{gateway_ip}' '{GATEWAY_HOSTNAME}' "
|
|
f"'{_MARKER}' >> /tmp/.bb-hosts; "
|
|
"cat /tmp/.bb-hosts > /etc/hosts; "
|
|
"rm -f /tmp/.bb-hosts"
|
|
)
|
|
|
|
|
|
def set_gateway_host(container_name: str, gateway_ip: str) -> None:
|
|
"""Point `GATEWAY_HOSTNAME` at `gateway_ip` inside one running container.
|
|
|
|
Must run before the agent is exec'd: the agent's proxy URL names the
|
|
gateway, so the entry has to exist for its first connection. Idempotent —
|
|
re-running with the same address is a no-op in effect.
|
|
"""
|
|
container_mod.exec_container_as_root(
|
|
container_name, ["sh", "-c", _rewrite_script(gateway_ip)],
|
|
)
|
|
|
|
|
|
def refresh_gateway_host(gateway_ip: str) -> list[str]:
|
|
"""Re-point every running bottle at the current gateway address.
|
|
|
|
Called once the shared gateway is known to be up, so a bottle stranded by
|
|
an earlier gateway restart re-attaches instead of needing a relaunch.
|
|
Returns the containers updated.
|
|
|
|
Best-effort per bottle: one container that refuses the write (already
|
|
exiting, say) must not stop the others from being repaired, and must not
|
|
fail the launch that triggered the sweep.
|
|
"""
|
|
updated: list[str] = []
|
|
for agent in enumerate_active():
|
|
name = f"{CONTAINER_NAME_PREFIX}{agent.slug}"
|
|
try:
|
|
set_gateway_host(name, gateway_ip)
|
|
updated.append(name)
|
|
# One bad bottle must not stop the sweep, so this is deliberately broad.
|
|
except Exception as e: # noqa: BLE001 # pylint: disable=broad-exception-caught
|
|
warn(f"could not re-point {name} at the gateway: {e}")
|
|
return updated
|
|
|
|
|
|
__all__ = ["GATEWAY_HOSTNAME", "set_gateway_host", "refresh_gateway_host"]
|