feat(firecracker): NAT'd egress link for the orchestrator/builder VM
The orchestrator/gateway VM is trusted infra, not an isolated agent: it builds agent images in-VM (buildah must FROM-pull + apt/npm) and, in the Stage B cutover, forwards agent egress upstream. Give it a dedicated TAP (`bborch0`) on a /31 at the top of the IP_BASE /16 (clear of the bbfc* agent pool at the bottom), NAT'd out the host uplink — while agent VMs keep their fail-closed, gateway-only isolation table. - netpool.defaults.env / netpool.py: new BOT_BOTTLE_FC_ORCH_IFACE + `orch_slot()` (index -1 sentinel; host x.y.255.0 / guest x.y.255.1). - scripts/firecracker-netpool.sh: create + address the orchestrator TAP; `bot_bottle_fc_nat` table masquerades its /31 out the uplink and accepts its forward path. Because bootstrap still runs Docker (whose FORWARD policy is DROP), a best-effort, guarded, idempotent DOCKER-USER ACCEPT is added too (skipped once Docker is gone). down/status updated. - nix/firecracker-netpool.nix: mirror the option, pass it via the unit Environment= (the store-copied script can't read the defaults file), and add iptables to the unit path for the DOCKER-USER step. Agent isolation is unchanged: the new rules only ever accept/masquerade the orchestrator link and never drop, so they can't weaken the bbfc* drops. Applied by re-running `sudo ./scripts/firecracker-netpool.sh up`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
@@ -49,10 +49,12 @@ let
|
||||
# /31 alignment == an even final octet (only bit 0 matters for base+2i).
|
||||
lastOctet = lib.toInt (lib.last (lib.splitString "." cfg.ipBase));
|
||||
|
||||
# The script needs ip/nft/sysctl + the usual coreutils. It gets every
|
||||
# pool value via the unit's Environment=, so it never reads the shared
|
||||
# defaults file (which isn't beside it once copied to the store).
|
||||
runtimePath = with pkgs; [ iproute2 nftables procps coreutils gnused ];
|
||||
# The script needs ip/nft/sysctl + the usual coreutils, plus iptables
|
||||
# for the orchestrator link's DOCKER-USER accept (best-effort; skipped
|
||||
# when Docker is absent). It gets every pool value via the unit's
|
||||
# Environment=, so it never reads the shared defaults file (which isn't
|
||||
# beside it once copied to the store).
|
||||
runtimePath = with pkgs; [ iproute2 nftables iptables procps coreutils gnused ];
|
||||
|
||||
ownEnv =
|
||||
if cfg.group != null
|
||||
@@ -64,6 +66,7 @@ let
|
||||
BOT_BOTTLE_FC_IP_BASE = cfg.ipBase;
|
||||
BOT_BOTTLE_FC_IFACE_PREFIX = cfg.ifacePrefix;
|
||||
BOT_BOTTLE_FC_NFT_TABLE = cfg.tableName;
|
||||
BOT_BOTTLE_FC_ORCH_IFACE = cfg.orchIface;
|
||||
} // ownEnv;
|
||||
in
|
||||
{
|
||||
@@ -127,6 +130,19 @@ in
|
||||
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
|
||||
};
|
||||
|
||||
orchIface = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = defaults.BOT_BOTTLE_FC_ORCH_IFACE;
|
||||
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
|
||||
description = ''
|
||||
TAP name for the orchestrator/gateway VM's dedicated link. Unlike
|
||||
the isolated bbfc* agent pool, this link is NAT'd to the internet
|
||||
(the orchestrator is trusted infra that builds agent images in-VM
|
||||
and forwards agent egress upstream). Must match
|
||||
BOT_BOTTLE_FC_ORCH_IFACE / netpool.ORCH_IFACE.
|
||||
'';
|
||||
};
|
||||
|
||||
writeEnvFile = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
@@ -176,6 +192,7 @@ in
|
||||
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
|
||||
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
|
||||
BOT_BOTTLE_FC_NFT_TABLE=${cfg.tableName}
|
||||
BOT_BOTTLE_FC_ORCH_IFACE=${cfg.orchIface}
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user