feat(firecracker): NAT'd egress link for the orchestrator/builder VM
lint / lint (push) Successful in 2m22s
test / unit (pull_request) Successful in 1m20s
test / integration (pull_request) Successful in 29s
test / coverage (pull_request) Successful in 1m27s

The orchestrator/gateway VM is trusted infra, not an isolated agent: it
builds agent images in-VM (buildah must FROM-pull + apt/npm) and, in the
Stage B cutover, forwards agent egress upstream. Give it a dedicated TAP
(`bborch0`) on a /31 at the top of the IP_BASE /16 (clear of the bbfc*
agent pool at the bottom), NAT'd out the host uplink — while agent VMs
keep their fail-closed, gateway-only isolation table.

- netpool.defaults.env / netpool.py: new BOT_BOTTLE_FC_ORCH_IFACE +
  `orch_slot()` (index -1 sentinel; host x.y.255.0 / guest x.y.255.1).
- scripts/firecracker-netpool.sh: create + address the orchestrator TAP;
  `bot_bottle_fc_nat` table masquerades its /31 out the uplink and
  accepts its forward path. Because bootstrap still runs Docker (whose
  FORWARD policy is DROP), a best-effort, guarded, idempotent DOCKER-USER
  ACCEPT is added too (skipped once Docker is gone). down/status updated.
- nix/firecracker-netpool.nix: mirror the option, pass it via the unit
  Environment= (the store-copied script can't read the defaults file),
  and add iptables to the unit path for the DOCKER-USER step.

Agent isolation is unchanged: the new rules only ever accept/masquerade
the orchestrator link and never drop, so they can't weaken the bbfc*
drops. Applied by re-running `sudo ./scripts/firecracker-netpool.sh up`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
2026-07-14 18:13:14 -04:00
parent bab44dbe97
commit 95981ea9d3
5 changed files with 149 additions and 5 deletions
+21 -4
View File
@@ -49,10 +49,12 @@ let
# /31 alignment == an even final octet (only bit 0 matters for base+2i).
lastOctet = lib.toInt (lib.last (lib.splitString "." cfg.ipBase));
# The script needs ip/nft/sysctl + the usual coreutils. It gets every
# pool value via the unit's Environment=, so it never reads the shared
# defaults file (which isn't beside it once copied to the store).
runtimePath = with pkgs; [ iproute2 nftables procps coreutils gnused ];
# The script needs ip/nft/sysctl + the usual coreutils, plus iptables
# for the orchestrator link's DOCKER-USER accept (best-effort; skipped
# when Docker is absent). It gets every pool value via the unit's
# Environment=, so it never reads the shared defaults file (which isn't
# beside it once copied to the store).
runtimePath = with pkgs; [ iproute2 nftables iptables procps coreutils gnused ];
ownEnv =
if cfg.group != null
@@ -64,6 +66,7 @@ let
BOT_BOTTLE_FC_IP_BASE = cfg.ipBase;
BOT_BOTTLE_FC_IFACE_PREFIX = cfg.ifacePrefix;
BOT_BOTTLE_FC_NFT_TABLE = cfg.tableName;
BOT_BOTTLE_FC_ORCH_IFACE = cfg.orchIface;
} // ownEnv;
in
{
@@ -127,6 +130,19 @@ in
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
};
orchIface = lib.mkOption {
type = lib.types.str;
default = defaults.BOT_BOTTLE_FC_ORCH_IFACE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = ''
TAP name for the orchestrator/gateway VM's dedicated link. Unlike
the isolated bbfc* agent pool, this link is NAT'd to the internet
(the orchestrator is trusted infra that builds agent images in-VM
and forwards agent egress upstream). Must match
BOT_BOTTLE_FC_ORCH_IFACE / netpool.ORCH_IFACE.
'';
};
writeEnvFile = lib.mkOption {
type = lib.types.bool;
default = false;
@@ -176,6 +192,7 @@ in
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
BOT_BOTTLE_FC_NFT_TABLE=${cfg.tableName}
BOT_BOTTLE_FC_ORCH_IFACE=${cfg.orchIface}
'';
};
};