Update Manifest.toml files with appropriate Julia versions

[tkf]

Now that CompatHelper updates all Manifest.toml by default, it would be great if it looks at julia compatibility bounds in Project.toml and uses appropriate binaries to do this (or alternatively Pkg.jl takes julia bound into account while Pkg.update). ATM, it's very hard to support julia < 1.3 and >= 1.3 while making CI reproducible by checking in Manifest.toml file as many packages start dropping julia < 1.3 due to the new artifacts support.

I'm thinking to check in e.g. test/environments/jl10/Manifest.toml with julia = "< 1.3" in Project.toml so that I can still test it with Julia 1.0 via Run.test(project="test/environments/jl10"). However this requires test/environments/jl10/Manifest.toml to be updated with julia 1.0 while test/Manifest.toml to be updated with the latest julia.

I suppose this would require Julia installer as a Julia package (e.g., using JuliaLang/julia#33817) and/or distributing julia binaries as artifacts?

[tkf]

For the moment, here is the setup I'm using to do this: https://github.com/tkf/BangBang.jl/blob/v0.3.12/.github/workflows/pkg-update.yml

[maleadt]

Example of where this behavior breaks CI: JuliaGPU/NCCL.jl#11 where Julia 1.0 does not support CuArrays 1.7.0 but CompatHelper happily updates the Manifest to use CuArrays 1.7.

I had to do something similar for my Pkg.update workflow, https://github.com/JuliaGPU/CUDAnative.jl/blob/3c9b2794c63eb8eb0029f52673f701bdd289e75c/.github/workflows/update_manifest.yml, where I take the julia bound as specified in the Project.toml (assuming it's a fully-specified version, otherwise the julia-setup action takes the most recent supported version).

[DilumAluthge]

Example of where this behavior breaks CI: JuliaGPU/NCCL.jl#11 where Julia 1.0 does not support CuArrays 1.7.0 but CompatHelper happily updates the Manifest to use CuArrays 1.7.

I had to do something similar for my Pkg.update workflow, https://github.com/JuliaGPU/CUDAnative.jl/blob/3c9b2794c63eb8eb0029f52673f701bdd289e75c/.github/workflows/update_manifest.yml, where I take the julia bound as specified in the Project.toml (assuming it's a fully-specified version, otherwise the julia-setup action takes the most recent supported version).

Why not just run CompatHelper on Julia 1.0? For example: JuliaGPU/NCCL.jl#16

[maleadt]

Why not just run CompatHelper on Julia 1.0? For example: JuliaGPU/NCCL.jl#16

Right, of course. And I can also combine that with the "grep julia compat" logic I already have.

[tkf]

CompatHelper.jl works with 1.0 now? That's great. Then maybe update the bound?

https://github.com/bcbi/CompatHelper.jl/blob/450d38f673b9d0a15060f02fa2c77436e3a4b52b/Project.toml#L19

But there are many packages ditching Julia < 1.3. You can't use 1.0 to update Manifest.toml if you want to test with the latest packages. (And without updating the manifest files, PRs created by CompatHelper.jl are not maximally useful.)

[DilumAluthge]

CompatHelper.jl works with 1.0 now? That's great. Then maybe update the bound?

Oh no, you’re right :(. It does require some features of Pkg only available in 1.1.

[DilumAluthge]

I mean, in my opinion, we should have multiple Manifests, one for each Julia version. E.g. Manifest.1.0.toml, Manifest.1.1.toml, Manifest.1.2.toml, etc. After all, a Manifest is only guaranteed to be compatible with the same version of Julia that generated it.

[tkf]

Yes, I'm doing it (or at least an approximation of it to workaround the lack of tooling).