Meta-ticket: CI workflows for upstream projects using Sage reusable workflows

[mkoeppe]

Deployments of the Sage portability CI and wheel-building workflows

and other integrations in upstream projects

Upstream project	Status	Update
4ti2/4ti2
dimpase/primecountpy
discopt/cmr
flintlib/python-flint		2024-02
ipython/ipython (sagemath-repl)		ipython/ipython#14317
kimwalisch/primecount		needs update
kimwalisch/primesieve	removed by upstream
latte-int/latte
linbox-team/fflas-ffpack		2024-01
mesonbuild/meson-python	replaced by handwritten Dockerfiles in mesonbuild/meson-python#241	2022-12
Normaliz/Normaliz		Normaliz/Normaliz#424
Normaliz/PyNormaliz
pypa/setuptools		pypa/setuptools#4203
sagemath/cypari2
sagemath/cysignals
sagemath/sage
sagemath/sage-numerical-backends-coin
scipopt/SCIP-SDP		scipopt/SCIP-SDP#9; needs updating after #37237 is merged
Singular/Singular		Singular/Singular#1229
sympy/sympy

• unmerged PRs to add:
  • CI: Add portability and integration tests with passagemath flintlib/flint#1760
  • CI: Add integration testing with Sage mpmath/mpmath#727
  • GH Actions workflow for continuous integration and portability testing fricas/fricas#102
  • ci-sage.yml: Use https://trac.sagemath.org/ticket/34106 flintlib/arb#434
  • CI: Portability tests for building on various Linux distributions and macOS scipopt/PySCIPOpt#630
  • .github/workflows/ci-sage.yml: New milahu/gnumake-tokenpool#12

---

Refactoring techniques:

• scripts in SAGE_ROOT/.ci/;
• new top-level SAGE_ROOT/Makefile targets;
• reusable workflows in the main Sage repo - https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
• reusable workflows in a different repo.
• problem matchers:
  • https://github.com/actions/toolkit/blob/main/docs/problem-matchers.md
  • https://github.com/actions/toolkit/blob/main/docs/commands.md#problem-matchers
  • Example: https://github.com/actions/setup-python/blob/main/.github/python.json
  • More examples: https://github.com/juftin/lunchable/tree/main/.github/matchers, https://github.com/python/cpython/tree/main/.github/problem-matchers

Tickets:

• Restore "huge" package type #29919
• ci-cygwin: Refactor using reusable workflows #33791
• tox.yml: Refactor using reusable workflows, update Linux platforms #34115
• CI: Merge open blocker PRs in all CI workflows + other improvements #36338

Part of:

• Meta-ticket: Refactor and improve system package related scripts, tox.ini, build/bin/write_dockerfile.sh #29146
• Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages #29060

References:

• Add integration testing with python from Debian, Ubuntu, Fedora, Gentoo, Arch Linux, OpenSUSE, Slackware pypa/setuptools#2923 (comment)

Depends on #33062

CC: @tobiasdiez @dimpase @saraedum

Component: porting

Issue created by migration from https://trac.sagemath.org/ticket/33338

[mkoeppe]

Description changed:

--- 
+++ 
@@ -15,4 +15,5 @@
 - reusable workflows in the main Sage repo - https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
 - reusable workflows in a different repo.
 
-
+References:
+-  https://github.com/pypa/setuptools/pull/2923#discussion_r795977169

[mkoeppe]

Description changed:

--- 
+++ 
@@ -15,5 +15,9 @@
 - reusable workflows in the main Sage repo - https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
 - reusable workflows in a different repo.
 
+Part of:
+- #29146 Meta-ticket: Refactor and improve system package related scripts, `tox.ini`, `build/bin/write_dockerfile.sh`
+- #29060 Meta-ticket: Add `Dockerfile`s and CI scripts for integration testing of source and binary distributions and of downstream packages
+
 References:
 -  https://github.com/pypa/setuptools/pull/2923#discussion_r795977169

[mkoeppe]

Description changed:

--- 
+++ 
@@ -7,6 +7,10 @@
 - https://github.com/4ti2/4ti2/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/wbhart/flint2/blob/flint-2.8/.github/workflows/ci-sage.yml
 - https://github.com/fredrik-johansson/arb/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/linbox-team/fflas-ffpack/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/kimwalisch/primesieve/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/kimwalisch/primecount/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/dimpase/primecountpy/tree/master/.github/workflows
 - ...
 
 We refactor them using a combination of 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -19,6 +19,9 @@
 - reusable workflows in the main Sage repo - https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
 - reusable workflows in a different repo.
 
+Tickets:
+- #29919 Restore "huge" package type
+
 Part of:
 - #29146 Meta-ticket: Refactor and improve system package related scripts, `tox.ini`, `build/bin/write_dockerfile.sh`
 - #29060 Meta-ticket: Add `Dockerfile`s and CI scripts for integration testing of source and binary distributions and of downstream packages

[tobiasdiez]

Description changed:

--- 
+++ 
@@ -18,6 +18,11 @@
 - new top-level `SAGE_ROOT/Makefile` targets; 
 - reusable workflows in the main Sage repo - https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
 - reusable workflows in a different repo.
+- problem matchers:
+  - https://github.com/actions/toolkit/blob/main/docs/problem-matchers.md
+  - https://github.com/actions/toolkit/blob/main/docs/commands.md#problem-matchers
+  - Example: https://github.com/actions/setup-python/blob/main/.github/python.json
+  - More examples: https://github.com/juftin/lunchable/tree/main/.github/matchers
 
 Tickets:
 - #29919 Restore "huge" package type

[tobiasdiez]

Description changed:

--- 
+++ 
@@ -22,7 +22,7 @@
   - https://github.com/actions/toolkit/blob/main/docs/problem-matchers.md
   - https://github.com/actions/toolkit/blob/main/docs/commands.md#problem-matchers
   - Example: https://github.com/actions/setup-python/blob/main/.github/python.json
-  - More examples: https://github.com/juftin/lunchable/tree/main/.github/matchers
+  - More examples: https://github.com/juftin/lunchable/tree/main/.github/matchers, https://github.com/python/cpython/tree/main/.github/problem-matchers
 
 Tickets:
 - #29919 Restore "huge" package type

[mkoeppe]

comment:8

#33791 does this for cygwin

[mkoeppe]

Description changed:

--- 
+++ 
@@ -11,6 +11,7 @@
 - https://github.com/kimwalisch/primesieve/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/kimwalisch/primecount/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/dimpase/primecountpy/tree/master/.github/workflows
+- https://github.com/latte-int/latte/blob/master/.github/workflows/ci-sage.yml
 - ...
 
 We refactor them using a combination of 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -13,6 +13,9 @@
 - https://github.com/dimpase/primecountpy/tree/master/.github/workflows
 - https://github.com/latte-int/latte/blob/master/.github/workflows/ci-sage.yml
 - ...
+
+PRs:
+- https://github.com/fricas/fricas/pull/102
 
 We refactor them using a combination of 
 - scripts in `SAGE_ROOT/.ci/`; 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -12,10 +12,13 @@
 - https://github.com/kimwalisch/primecount/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/dimpase/primecountpy/tree/master/.github/workflows
 - https://github.com/latte-int/latte/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/sagemath/cysignals/blob/main/.github/workflows/ci-sage.yml
 - ...
 
 PRs:
 - https://github.com/fricas/fricas/pull/102
+- https://github.com/fredrik-johansson/arb/pull/434
+- https://github.com/wbhart/flint2/pull/1166
 
 We refactor them using a combination of 
 - scripts in `SAGE_ROOT/.ci/`; 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -12,7 +12,6 @@
 - https://github.com/kimwalisch/primecount/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/dimpase/primecountpy/tree/master/.github/workflows
 - https://github.com/latte-int/latte/blob/master/.github/workflows/ci-sage.yml
-- https://github.com/sagemath/cysignals/blob/main/.github/workflows/ci-sage.yml
 - ...
 
 PRs:

[mkoeppe]

Description changed:

--- 
+++ 
@@ -32,6 +32,8 @@
 
 Tickets:
 - #29919 Restore "huge" package type
+- #33791 ci-cygwin: Refactor using reusable workflows
+- #34115 tox.yml: Refactor using reusable workflows
 
 Part of:
 - #29146 Meta-ticket: Refactor and improve system package related scripts, `tox.ini`, `build/bin/write_dockerfile.sh`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -18,6 +18,7 @@
 - https://github.com/fricas/fricas/pull/102
 - https://github.com/fredrik-johansson/arb/pull/434
 - https://github.com/wbhart/flint2/pull/1166
+- https://github.com/FFY00/meson-python/pull/133
 
 We refactor them using a combination of 
 - scripts in `SAGE_ROOT/.ci/`; 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -12,13 +12,13 @@
 - https://github.com/kimwalisch/primecount/blob/master/.github/workflows/ci-sage.yml
 - https://github.com/dimpase/primecountpy/tree/master/.github/workflows
 - https://github.com/latte-int/latte/blob/master/.github/workflows/ci-sage.yml
+- https://github.com/FFY00/meson-python/blob/main/.github/workflows/ci-sage.yml
 - ...
 
 PRs:
 - https://github.com/fricas/fricas/pull/102
 - https://github.com/fredrik-johansson/arb/pull/434
 - https://github.com/wbhart/flint2/pull/1166
-- https://github.com/FFY00/meson-python/pull/133
 
 We refactor them using a combination of 
 - scripts in `SAGE_ROOT/.ci/`; 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -19,6 +19,8 @@
 - https://github.com/fricas/fricas/pull/102
 - https://github.com/fredrik-johansson/arb/pull/434
 - https://github.com/wbhart/flint2/pull/1166
+- https://github.com/scipopt/PySCIPOpt/pull/630
+- https://github.com/scipopt/SCIP-SDP/pull/4
 
 We refactor them using a combination of 
 - scripts in `SAGE_ROOT/.ci/`;