Skip to content

GH-47797: [CI][Python] Update Python installs for free-threaded wheel tasks #47993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
raulcd merged 26 commits into from
Dec 19, 2025
Merged

GH-47797: [CI][Python] Update Python installs for free-threaded wheel tasks #47993

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
892b855
Update musllinux installs
AlenkaF Oct 28, 2025
603f75b
Use default args
AlenkaF Oct 30, 2025
b6d5b69
Update windows wheels dockerfiles to use MSIX
AlenkaF Nov 5, 2025
ddc9ae0
Remove free threaded win wheels from services (only build, not test -…
AlenkaF Nov 5, 2025
c901361
Bring bach win free-threaded service and only use the same dockerfile
AlenkaF Nov 5, 2025
c1ac11d
Update windows test wheels dockerfiles to use MSIX
AlenkaF Nov 5, 2025
d6ffdd3
Try moving install of MSIX to base dockerfile
AlenkaF Nov 5, 2025
836718a
Add MSIX to the test base dockerfile also
AlenkaF Nov 6, 2025
b773837
Use proper powershell syntax
AlenkaF Nov 6, 2025
8dbd304
Try if curl works
AlenkaF Nov 6, 2025
e9d67d5
Move curl command outside Poweshell
AlenkaF Nov 6, 2025
ecda84d
Change url and decouple choco and pymanager install
AlenkaF Nov 7, 2025
bf5c373
Revert "Update windows test wheels dockerfiles to use MSIX"
AlenkaF Nov 7, 2025
d13d7dd
Remove pymanager install from base test dockerfile
AlenkaF Nov 7, 2025
730729f
Revert unnecessary changes on the choco install in base test
AlenkaF Nov 7, 2025
59dce60
Fix wrong tab
AlenkaF Nov 7, 2025
834d8cb
Update ci/docker/python-wheel-windows-vs2022-base.dockerfile
AlenkaF Nov 10, 2025
080f9a4
Remove duplicated rows
AlenkaF Nov 10, 2025
258bf1e
Try removing defaults already parametered
AlenkaF Dec 3, 2025
52ac61e
Remove mapping to patch versions
AlenkaF Dec 3, 2025
0450baf
Include python_variant suggestion
AlenkaF Dec 3, 2025
2c9e532
Add build_date arg
AlenkaF Dec 4, 2025
b0d30cb
Fix win dockerfile
AlenkaF Dec 5, 2025
b4b42fb
Fix error - win test still using choco not pymanager
AlenkaF Dec 5, 2025
a356a46
Fix typo
AlenkaF Dec 5, 2025
cf0c169
Remove old comment
AlenkaF Dec 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update windows wheels dockerfiles to use MSIX
  • Loading branch information
@AlenkaF
AlenkaF committed Dec 18, 2025
commit b6d5b69f4e7bc8e316e78f65c20bfa7b04bc9dee
53 changes: 0 additions & 53 deletions ci/docker/python-free-threaded-wheel-windows-vs2022.dockerfile
View file
Open in desktop

This file was deleted.

17 changes: 16 additions & 1 deletion ci/docker/python-wheel-windows-vs2022.dockerfile
View file
Open in desktop
Copy link
Member

@pitrou pitrou Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do similar changes for python-wheel-windows-test-vs2022.dockerfile and friends?

Copy link
Member Author

@AlenkaF AlenkaF Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to make it work for the test group. Looks like a different underlying image might not be able to support this kind of pymanager install, but have no experience really =)

More info on what issues I had:

Copy link
Member

@pitrou pitrou Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, fair enough. Can you open an issue so that it can be investigated later?

Copy link
Member Author

@AlenkaF AlenkaF Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: #48469

Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,22 @@ RUN (if "%python%"=="3.10" setx PYTHON_VERSION "3.10.11" && setx PYTHON_CMD "py
(if "%python%"=="3.13" setx PYTHON_VERSION "3.13.9" && setx PYTHON_CMD "py -3.13") & \
(if "%python%"=="3.14" setx PYTHON_VERSION "3.14.0" && setx PYTHON_CMD "py -3.14")

RUN choco install -r -y --pre --no-progress python --version=%PYTHON_VERSION%
SHELL ["powershell", "-NoProfile", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
# Install Python install manager (MSIX)
RUN $msix_url = 'https://www.python.org/ftp/python/installer/python.msix'; \
Invoke-WebRequest -Uri $msix_url -OutFile 'python.msix'; \
Add-AppxPackage .\python.msix

# Use python_abi_tag env var to select regular or free-threaded Python
ARG python_abi_tag=cp310
ENV PYTHON_ABI_TAG=${python_abi_tag}
RUN if ($env:PYTHON_ABI_TAG -match 't$') { \
pymanager install --version $env:PYTHON_VERSION --variant freethreaded \
} else { \
pymanager install --version $env:PYTHON_VERSION \
}

SHELL ["cmd", "/S", "/C"]
RUN %PYTHON_CMD% -m pip install -U pip setuptools

COPY python/requirements-wheel-build.txt C:/arrow/python/
Expand Down
17 changes: 1 addition & 16 deletions compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1388,6 +1388,7 @@ services:
args:
base: ${REPO}:python-wheel-windows-vs2022-base-vcpkg-${VCPKG}-${PYTHON_WHEEL_WINDOWS_IMAGE_REVISION}
python: ${PYTHON}
python_abi_tag: ${PYTHON_ABI_TAG}
context: .
dockerfile: ci/docker/python-wheel-windows-vs2022.dockerfile
# This should make the pushed images reusable, but the image gets rebuilt.
Expand All @@ -1398,22 +1399,6 @@ services:
volumes: *python-wheel-windows-vs2022-volumes
command: arrow\\ci\\scripts\\python_wheel_windows_build.bat

python-free-threaded-wheel-windows-vs2022:
image: ${REPO}:python-${PYTHON}-free-threaded-wheel-windows-vs2022-vcpkg-${VCPKG}-${PYTHON_WHEEL_WINDOWS_IMAGE_REVISION}
build:
args:
base: ${REPO}:python-wheel-windows-vs2022-base-vcpkg-${VCPKG}-${PYTHON_WHEEL_WINDOWS_IMAGE_REVISION}
python: ${PYTHON}
context: .
dockerfile: ci/docker/python-free-threaded-wheel-windows-vs2022.dockerfile
# This should make the pushed images reusable, but the image gets rebuilt.
# Uncomment if no local cache is available.
# cache_from:
# - mcr.microsoft.com/windows/servercore:ltsc2022
# - ${REPO}:python-${PYTHON}-free-threaded-wheel-windows-vs2022-vcpkg-${VCPKG}-${PYTHON_WHEEL_WINDOWS_IMAGE_REVISION}
volumes: *python-wheel-windows-vs2022-volumes
command: arrow\\ci\\scripts\\python_wheel_windows_build.bat

python-wheel-windows-test-base:
image: ${REPO}:python-wheel-windows-test-vs2022-base-${PYTHON_WHEEL_WINDOWS_TEST_IMAGE_REVISION}
build:
Expand Down