perf: lazy-load backend modules and consolidate docker subprocess helpers #383

2026-07-14T04:21:58-04:00

didericis-claude commented

2026-07-14 04:21:58 -04:00

Closes #359.

Summary

backend/__init__.py previously imported all three concrete backends at module load time, causing backend.docker.util to drag in ~76 modules (firecracker, macOS container, manifest, egress, git-gate, supervise, workspace). This PR makes those imports lazy via _get_backends() and module-level __getattr__, so they only run when a backend is first used.
backend/docker/util.py subprocess helpers (runsc_available, image_exists, container_exists, force_remove_container, commit_container) now go through the shared run_docker primitive instead of raw subprocess.run calls, removing the local _silent_run shim.
Added inline # pylint: disable annotations for the two false positives the lazy pattern introduces (global-statement on the singleton setter, undefined-all-variable on __all__ entries exposed via __getattr__).

Closes #359. ## Summary - `backend/__init__.py` previously imported all three concrete backends at module load time, causing `backend.docker.util` to drag in ~76 modules (firecracker, macOS container, manifest, egress, git-gate, supervise, workspace). This PR makes those imports lazy via `_get_backends()` and module-level `__getattr__`, so they only run when a backend is first used. - `backend/docker/util.py` subprocess helpers (`runsc_available`, `image_exists`, `container_exists`, `force_remove_container`, `commit_container`) now go through the shared `run_docker` primitive instead of raw `subprocess.run` calls, removing the local `_silent_run` shim. - Added inline `# pylint: disable` annotations for the two false positives the lazy pattern introduces (`global-statement` on the singleton setter, `undefined-all-variable` on `__all__` entries exposed via `__getattr__`).

~~didericis-claude referenced this pull request 2026-07-14 04:38:57 -04:00~~

feat(firecracker): consolidated per-host gateway launch (PRD 0070, part of #348) #354

didericis force-pushed lazy-backend-imports from b05775b581 to 985e79bd8e

2026-07-18 04:58:18 -04:00

Compare

didericis-codex approved these changes 2026-07-18 05:01:52 -04:00

didericis-codex left a comment

Looks good. I found no correctness issues in the lazy backend loading or the Docker subprocess-helper consolidation. Package-level compatibility is preserved through getattr, and the migrated Docker calls retain their intended return-code/error handling.

Validated locally: focused backend-selection and Docker-util tests pass (19/19), and importing bot_bottle.backend no longer loads the concrete backend modules. The current red lint/unit checks appear unrelated to this change: both runners fail during dependency installation with PEP 668 externally-managed-environment errors; integration and coverage are green.

Looks good. I found no correctness issues in the lazy backend loading or the Docker subprocess-helper consolidation. Package-level compatibility is preserved through __getattr__, and the migrated Docker calls retain their intended return-code/error handling. Validated locally: focused backend-selection and Docker-util tests pass (19/19), and importing bot_bottle.backend no longer loads the concrete backend modules. The current red lint/unit checks appear unrelated to this change: both runners fail during dependency installation with PEP 668 externally-managed-environment errors; integration and coverage are green.

didericis force-pushed lazy-backend-imports from 985e79bd8e to 78d72264c8

2026-07-18 05:03:01 -04:00

Compare

didericis added 3 commits 2026-07-18 05:11:34 -04:00

perf: lazy-load backend modules and consolidate docker subprocess helpers 3fe90b3a88

Importing backend.docker.util previously triggered eager loading of all
three backend packages (~76 modules) because backend/__init__.py imported
DockerBottleBackend, FirecrackerBottleBackend, and MacosContainerBottleBackend
at module scope. This made the module prohibitively expensive to import
from the orchestrator layer and elsewhere.

The three backend imports are now deferred into _get_backends(), which
loads all three on first call and caches the result in the module-level
_BACKENDS variable (initially None). Module-level __getattr__ exposes
backend classes and freeze symbols lazily for existing import/patch sites.

backend/docker/util.py raw subprocess.run(["docker", ...]) calls are
replaced with the shared run_docker primitive from docker_cmd, eliminating
the duplication between the backend and orchestrator implementations.
_silent_run() is removed; image_exists() is inlined directly onto
run_docker. The commit_container test is updated to patch run_docker
instead of subprocess.run.

fix(backend): silence pylint false positives from lazy-load pattern 4b8b09ab83

`undefined-all-variable` fires on CommitCancelled / Freezer / get_freezer
in __all__ because pylint can't see module-level __getattr__ bindings;
`global-statement` fires on the _BACKENDS singleton setter. Both are
intentional patterns — add inline disables rather than suppress globally.

fix(backend): fix pyright errors in lazy-load implementation

test / integration (pull_request) Successful in 12s

Details

test / unit (pull_request) Successful in 32s

Details

test / coverage (pull_request) Successful in 36s

Details

lint / lint (push) Successful in 2m17s

Details

8a321770dd

- Rename _BACKENDS → _backends: pyright treats uppercase module-level
  names as constants and flags the reassignment in _get_backends() as
  reportConstantRedefinition; lowercase avoids this.
- Add TYPE_CHECKING guard importing CommitCancelled/Freezer/get_freezer
  from .freeze: pyright cannot see module-level __getattr__ bindings, so
  reportUnsupportedDunderAll fired for those three __all__ entries; the
  guard makes them visible to the type checker without running at import
  time.
- Update test_backend_selection.py to patch _backends (lowercase).

didericis force-pushed lazy-backend-imports from 78d72264c8 to 8a321770dd

2026-07-18 05:11:34 -04:00

Compare

didericis merged commit 410c19aaaf into main

2026-07-18 05:14:28 -04:00

didericis deleted branch lazy-backend-imports

2026-07-18 05:14:29 -04:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: didericis/bot-bottle#383