Summary
The Linux x86_64 cp312 wheel is now too large to upload to PyPI.
Concrete blocker:
- Wheel:
envpool-1.1.0-cp312-cp312-manylinux_2_28_x86_64.whl
- Current size:
117,913,151 bytes
- PyPI project file-size limit for
envpool: 100 MB
- Upload failure:
400 File too large. Limit for project 'envpool' is 100 MB
This means the current release workflow can successfully build the wheel, but the PyPI publish step cannot ship it.
Current observations
A few measurements from the 1.1.0 Linux cp312 x86_64 wheel:
- Running
strip --strip-unneeded on the EnvPool-built Linux .so files only reduces the wheel from 117,913,151 to 115,440,654 bytes, so strip-only is not enough.
- The biggest contributors include:
envpool/vizdoom/bin/freedoom2.wad (28.5 MB raw)
envpool.libs/libicudata-*.so (26.9 MB raw)
- large vendored Qt/ICU libraries pulled into
envpool.libs
mujoco/robotics/assets/**/* (~25.9 MB compressed total)
- There are also confirmed dead robotics assets in the current wheel that are not referenced by the currently registered robotics envs, worth about
~12.7 MB compressed in total.
Possible optimization directions
This issue is intentionally not prescribing a single fix yet. Reasonable directions include:
- Strip Linux-built shared objects in the release build.
- This should be done regardless, but it is only a partial improvement.
- Remove confirmed unused robotics assets from the wheel.
- There are currently dead files under the robotics asset tree that appear to be unused by the registered env set.
- This is a relatively low-risk size win.
- Move large third-party assets out of the core wheel.
- Candidates include ViZDoom runtime assets (
freedoom2.wad, vizdoom, maps) and large MuJoCo robotics assets.
- Possible implementations:
- runtime download with hash verification and local cache
- explicit prefetch command
- split asset package(s)
- Reduce or isolate the Linux-only vendored dependency payload.
- In particular,
procgen currently pulls Qt/ICU into envpool.libs on Linux.
- If possible, either make that dependency smaller, make it optional, or split it out of the base wheel.
CI follow-up requested in this issue
After we have a fix or mitigation, we should add a dedicated wheel-size guard in CI:
- Run the Linux
x86_64 py3.12 release-wheel build on every commit / PR update, not only on main.
- Do not run the full release matrix for this guard; only run the single
release-linux (x86_64, py3.12) build.
- After building
wheelhouse/*.whl, assert that the produced wheel is strictly below PyPI's limit (< 100 MB).
- Make that check fail loudly so regressions are caught before release time.
Why this matters
Right now, the failure is only discovered at release/publish time. A dedicated per-commit size guard for the exact problematic target would make this a normal CI regression instead of a release-day surprise.
Summary
The Linux
x86_64cp312wheel is now too large to upload to PyPI.Concrete blocker:
envpool-1.1.0-cp312-cp312-manylinux_2_28_x86_64.whl117,913,151bytesenvpool:100 MB400 File too large. Limit for project 'envpool' is 100 MBThis means the current release workflow can successfully build the wheel, but the PyPI publish step cannot ship it.
Current observations
A few measurements from the
1.1.0Linuxcp312 x86_64wheel:strip --strip-unneededon the EnvPool-built Linux.sofiles only reduces the wheel from117,913,151to115,440,654bytes, so strip-only is not enough.envpool/vizdoom/bin/freedoom2.wad(28.5 MBraw)envpool.libs/libicudata-*.so(26.9 MBraw)envpool.libsmujoco/robotics/assets/**/*(~25.9 MBcompressed total)~12.7 MBcompressed in total.Possible optimization directions
This issue is intentionally not prescribing a single fix yet. Reasonable directions include:
freedoom2.wad,vizdoom, maps) and large MuJoCo robotics assets.procgencurrently pulls Qt/ICU intoenvpool.libson Linux.CI follow-up requested in this issue
After we have a fix or mitigation, we should add a dedicated wheel-size guard in CI:
x86_64py3.12release-wheel build on every commit / PR update, not only onmain.release-linux (x86_64, py3.12)build.wheelhouse/*.whl, assert that the produced wheel is strictly below PyPI's limit (< 100 MB).Why this matters
Right now, the failure is only discovered at release/publish time. A dedicated per-commit size guard for the exact problematic target would make this a normal CI regression instead of a release-day surprise.