didericis/bot-bottle
1
0
Fork 0
Code Issues 34 Pull Requests 8 Actions 2 Packages Releases Wiki Activity

refactor(gateway): replace flat-file import shims with installed package #410

Merged
didericis merged 4 commits from fix/gateway-bundle-packaging into main 2026-07-18 05:02:42 -04:00
Conversation 3 Commits 4 Files Changed 16
+113 -214
didericis-claude commented 2026-07-17 22:54:21 -04:00
Collaborator
Copy Link
Copy Source

Closes #400 (item 4).

Summary

  • Adds pyproject.toml so pip install . works, and replaces 14 flat COPY statements in Dockerfile.gateway with COPY bot_bottle/ /src/bot_bottle/ && pip install /src/ plus 4 entry-point copies — missing or renamed modules now fail at image build time rather than container runtime.
  • Adds bot_bottle/constants.py as a single source of truth for IDENTITY_HEADER and GIT_GATE_TIMEOUT_SECS, removing the duplicated literal definitions from egress_addon.py, supervise_server.py, git_http_backend.py, and git_gate_render.py.
  • Removes all try/except import shims from egress_addon_core, dlp_detectors, egress_addon, supervise, supervise_server, and git_http_backend — every import is now a plain from bot_bottle.X import Y.
  • Test files updated to match: test_supervise_server.py drops the sys.path.insert hack; both test_egress_addon_* shim files drop the sys.modules["egress_addon_core"] pre-population.
Closes #400 (item 4). ## Summary - Adds `pyproject.toml` so `pip install .` works, and replaces 14 flat `COPY` statements in `Dockerfile.gateway` with `COPY bot_bottle/ /src/bot_bottle/ && pip install /src/` plus 4 entry-point copies — missing or renamed modules now fail at image build time rather than container runtime. - Adds `bot_bottle/constants.py` as a single source of truth for `IDENTITY_HEADER` and `GIT_GATE_TIMEOUT_SECS`, removing the duplicated literal definitions from `egress_addon.py`, `supervise_server.py`, `git_http_backend.py`, and `git_gate_render.py`. - Removes all `try/except` import shims from `egress_addon_core`, `dlp_detectors`, `egress_addon`, `supervise`, `supervise_server`, and `git_http_backend` — every import is now a plain `from bot_bottle.X import Y`. - Test files updated to match: `test_supervise_server.py` drops the `sys.path.insert` hack; both `test_egress_addon_*` shim files drop the `sys.modules["egress_addon_core"]` pre-population.
didericis added 1 commit 2026-07-17 23:01:49 -04:00
refactor(gateway): replace flat-file import shims with installed package
lint / lint (push) Successful in 2m22s
Details
test / unit (pull_request) Successful in 1m12s
Details
test / integration (pull_request) Successful in 23s
Details
test / coverage (pull_request) Successful in 1m23s
Details
5ad3449e3b 
Install bot_bottle via pip in Dockerfile.gateway instead of COPYing
individual .py files flat under /app/. This eliminates the try/except
import shims in egress_addon_core, dlp_detectors, egress_addon,
supervise, supervise_server, and git_http_backend that existed only
to support the flat-bundle layout.

Adds bot_bottle/constants.py as a single source of truth for
IDENTITY_HEADER and GIT_GATE_TIMEOUT_SECS, removing the duplicated
literal definitions in egress_addon.py, supervise_server.py,
git_http_backend.py, and git_gate_render.py.

Test files updated to match: test_supervise_server.py drops the
sys.path.insert hack in favour of direct package imports; the
egress_addon test shims no longer pre-populate sys.modules with a
bare egress_addon_core alias.
didericis force-pushed fix/gateway-bundle-packaging from bed648522c to 5ad3449e3b 2026-07-17 23:01:49 -04:00 Compare
didericis reviewed 2026-07-18 03:48:18 -04:00
Dockerfile.gateway Outdated
@@ -107,4 +105,3 @@
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/gateway_init.py /app/gateway_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
didericis commented 2026-07-18 03:48:18 -04:00
Owner
Copy Link
Copy Source

Why are these still included? Didn't we add them via the new copy line?

Why are these still included? Didn't we add them via the new copy line?
didericis commented 2026-07-18 03:53:19 -04:00
Owner
Copy Link
Copy Source

We should probably just turn those into command that we can invoke from the package, right? Copying those again when they're in the package seems non ideal. Push back and correct me if I'm wrong. Also if we need a light shim that just does from bot_bottle import supervise_server; supervise_server()/something like that, that'd be fine.

We should probably just turn those into command that we can invoke from the package, right? Copying those again when they're in the package seems non ideal. Push back and correct me if I'm wrong. Also if we need a light shim that just does `from bot_bottle import supervise_server; supervise_server()`/something like that, that'd be fine.
didericis commented 2026-07-18 03:58:01 -04:00
Owner
Copy Link
Copy Source

Why can't we add a shim for that fourth case too, though? EX: mitmdump -s /app/egress_addon_shim.py, and then just have something like this in egress_addon_shim.py:

from bot_bottle import egress_addon
egress_addon()

Alternative would be to move egress_addon out of the bot_bottle package: still think having the file copied in two places is bad.

Why can't we add a shim for that fourth case too, though? EX: `mitmdump -s /app/egress_addon_shim.py`, and then just have something like this in `egress_addon_shim.py`: ```py from bot_bottle import egress_addon egress_addon() ``` Alternative would be to move `egress_addon` out of the bot_bottle package: still think having the file copied in two places is bad.
didericis marked this conversation as resolved
didericis-claude reviewed 2026-07-18 03:50:03 -04:00
Dockerfile.gateway Outdated
@@ -107,4 +105,3 @@
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/gateway_init.py /app/gateway_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
didericis-claude commented 2026-07-18 03:50:02 -04:00
Author
Collaborator
Copy Link
Copy Source

The COPY bot_bottle/ /src/bot_bottle/ feeds pip install /src/, which installs the package into site-packages as bot_bottle.*. The individual COPY lines here put the same files at /app/ — the fixed paths gateway_init.py uses when spawning the daemons as subprocesses (e.g. python3 /app/supervise_server.py) and that mitmdump uses for its -s /app/egress_addon.py argument. The two destinations serve different roles: one is the importable package, the other is the executable entry points.

The `COPY bot_bottle/ /src/bot_bottle/` feeds `pip install /src/`, which installs the package into site-packages as `bot_bottle.*`. The individual `COPY` lines here put the same files at `/app/` — the fixed paths `gateway_init.py` uses when spawning the daemons as subprocesses (e.g. `python3 /app/supervise_server.py`) and that mitmdump uses for its `-s /app/egress_addon.py` argument. The two destinations serve different roles: one is the importable package, the other is the executable entry points.
didericis-claude reviewed 2026-07-18 03:54:48 -04:00
Dockerfile.gateway Outdated
@@ -107,4 +105,3 @@
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/gateway_init.py /app/gateway_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
didericis-claude commented 2026-07-18 03:54:48 -04:00
Author
Collaborator
Copy Link
Copy Source

Good call — three of the four can go. supervise_server.py, git_http_backend.py, and gateway_init.py all have if __name__ == "__main__": guards, so python3 -m bot_bottle.X works and we can drop those COPY lines. The one that has to stay is egress_addon.py: egress-entrypoint.sh runs mitmdump -s /app/egress_addon.py, and mitmdump's -s flag takes a file path — there's no module equivalent. egress-entrypoint.sh itself also needs to stay for the same reason. Implementing the rest now.

Good call — three of the four can go. `supervise_server.py`, `git_http_backend.py`, and `gateway_init.py` all have `if __name__ == "__main__":` guards, so `python3 -m bot_bottle.X` works and we can drop those COPY lines. The one that has to stay is `egress_addon.py`: `egress-entrypoint.sh` runs `mitmdump -s /app/egress_addon.py`, and mitmdump's `-s` flag takes a file path — there's no module equivalent. `egress-entrypoint.sh` itself also needs to stay for the same reason. Implementing the rest now.
didericis added 1 commit 2026-07-18 03:55:44 -04:00
refactor(gateway): invoke daemons via python3 -m instead of /app/ file copies
lint / lint (push) Successful in 2m20s
Details
test / unit (pull_request) Successful in 1m12s
Details
test / integration (pull_request) Successful in 26s
Details
test / coverage (pull_request) Successful in 1m29s
Details
9a0dd821ef 
supervise_server, git_http_backend, and gateway_init all have __main__
guards, so python3 -m bot_bottle.X replaces the individual COPY lines
to /app/. egress_addon.py stays as a file copy because mitmdump -s
requires a file path rather than a module reference.
didericis added 1 commit 2026-07-18 04:00:06 -04:00
refactor(gateway): replace egress_addon.py copy with a one-line shim
test / integration (pull_request) Successful in 1m15s
Details
test / coverage (pull_request) Failing after 1m13s
Details
test / unit (pull_request) Successful in 1m19s
Details
232dfdf37a 
mitmdump -s requires a file path, not a module. Instead of copying the
full egress_addon.py to /app/, write a one-line shim at image build time
that re-exports addons from the installed package. mitmdump finds the
addons list in the shim's namespace; all real addon code stays in
bot_bottle/egress_addon.py.
didericis-claude reviewed 2026-07-18 04:00:16 -04:00
Dockerfile.gateway
didericis-claude commented 2026-07-18 04:00:15 -04:00
Author
Collaborator
Copy Link
Copy Source

You're right, implemented. mitmdump finds addons in whatever module -s points at, so a one-liner is enough:

from bot_bottle.egress_addon import addons

Generated inline with RUN printf ... > /app/egress_addon.py so no extra source file needed. egress-entrypoint.sh still points at /app/egress_addon.py, unchanged.

You're right, implemented. mitmdump finds `addons` in whatever module `-s` points at, so a one-liner is enough: ```python from bot_bottle.egress_addon import addons ``` Generated inline with `RUN printf ... > /app/egress_addon.py` so no extra source file needed. `egress-entrypoint.sh` still points at `/app/egress_addon.py`, unchanged.
didericis marked this conversation as resolved
didericis added 1 commit 2026-07-18 04:58:48 -04:00
ci(test): drop actions/setup-python; install into the container's system Python
test / integration (pull_request) Successful in 7s
Details
test / unit (pull_request) Successful in 44s
Details
test / coverage (pull_request) Successful in 36s
Details
test / integration (push) Successful in 7s
Details
Update Quality Badges / update-badges (push) Failing after 11s
Details
test / unit (push) Successful in 29s
Details
test / coverage (push) Successful in 35s
Details
lint / lint (push) Successful in 2m24s
Details
d3c4fc0fd4 
The old act_runner engine (v0.2.13 on the delphi-ci runner) mishandles
actions/setup-python's PATH injection: pip installs coverage into the
toolcache interpreter while `python3` in later steps resolves back to the
image's system Python, so unit/coverage jobs failed with "No module named
coverage". Newer runners (TrueNAS's v0.6.1) don't, which is why it only
broke on delphi.

The runner-images/act container already ships Python 3.12, and the job
container is ephemeral, so drop setup-python entirely and install straight
into the system Python with --break-system-packages. Every step now uses
one interpreter consistently, on any runner version. Also removes the
redundant setup-python step from the integration job (stdlib-only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
didericis approved these changes 2026-07-18 05:02:19 -04:00
didericis merged commit d3c4fc0fd4 into main 2026-07-18 05:02:42 -04:00
didericis deleted branch fix/gateway-bundle-packaging 2026-07-18 05:02:43 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
didericis-claude
didericis
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status/Needs Triage
Awaiting initial classification
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
didericis-claude (didericis (claude)) didericis-codex (didericis (codex)) didericis-gemma (gemma) didericis
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: didericis/bot-bottle#410
Delete Branch "fix/gateway-bundle-packaging"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?