fix: make orchestrator teardown timeout configurable (closes #435)
test / stage-firecracker-inputs (pull_request) Successful in 3s
lint / lint (push) Successful in 45s
test / unit (pull_request) Successful in 30s
test / integration-docker (pull_request) Successful in 7s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / build-infra (pull_request) Successful in 3m53s
test / integration-firecracker (pull_request) Successful in 1m36s
test / coverage (pull_request) Failing after 2m41s
test / publish-infra (pull_request) Has been skipped

Resolves via ENV VAR -> orchestrator DB config -> default (30 s, up from 5 s):
  BOT_BOTTLE_ORCHESTRATOR_TEARDOWN_TIMEOUT_SECONDS
  teardown_timeout_seconds key in new orchestrator_config table (bot-bottle.db)

New OrchestratorConfigStore (same DbStore/TableMigrations pattern as the
registry) stores the DB-level setting. resolve_teardown_timeout() implements
the priority chain and is called at stack.callback registration time in all
three backends (macos_container, docker, firecracker).
This commit is contained in:
2026-07-20 19:36:41 +00:00
parent 44479f328e
commit 660951baf5
7 changed files with 146 additions and 7 deletions
@@ -91,12 +91,18 @@ def launch_consolidated(
)
def teardown_consolidated(bottle_id: str, *, orchestrator_url: str) -> None:
def teardown_consolidated(
bottle_id: str, *, orchestrator_url: str, timeout: float | None = None,
) -> None:
"""Deregister the bottle and remove its git-gate state from the gateway
VM. Both steps are idempotent so this is safe from a cleanup trap. Does
NOT stop the infra VM — it's a persistent per-host singleton shared by
every bottle."""
OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id)
from ...orchestrator.config_store import DEFAULT_TEARDOWN_TIMEOUT_SECONDS
OrchestratorClient(
orchestrator_url,
timeout=timeout if timeout is not None else DEFAULT_TEARDOWN_TIMEOUT_SECONDS,
).teardown_bottle(bottle_id)
deprovision_git_gate(infra_vm.gateway_transport(), bottle_id)
+2
View File
@@ -52,6 +52,7 @@ from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import firecracker_vm, image_builder, isolation_probe, netpool, util
from .bottle import FirecrackerBottle
from .bottle_plan import FirecrackerBottlePlan
from ...orchestrator.config_store import resolve_teardown_timeout
from .consolidated_launch import (
launch_consolidated,
teardown_consolidated,
@@ -121,6 +122,7 @@ def launch(
stack.callback(
teardown_consolidated, ctx.bottle_id,
orchestrator_url=ctx.orchestrator_url,
timeout=resolve_teardown_timeout(),
)
# Step 5: install the SHARED gateway CA (replaces the per-bottle CA).