Fix cross compilation with clang-cl (v2)

[Sporif]

Supersedes #101.

This PR also fixes cross compilation of the C bindings and adds CI testing for 'Linux - Windows MSVC', but skips the 'C bindings intrinsics' test as that fails to build).

Successful build: https://github.com/Sporif/BLAKE3/actions/runs/1944217647

[oconnor663]

Wow, thank you for figuring all this out. It's probably time for me to refactor all of these copy-pasted jobs into a Python script or something, so that additions like this aren't so painful in the future.

[oconnor663]

@sneves are there any potential ABI mismatch issues we might need to worry about with compiling the Windows GNU .S files into a binary that's otherwise targeting the MSVC ABI? Like different callee-saved register conventions or whatever? Or is it just the assembly syntax that's different between the two?

[sneves]

Syntax only, yes.

[ark0f]

Any updates?

[tbillington]

Hey @oconnor663, just wanted to check if there was any chance of this PR being adopted/revisited?

I started applying the changes fresh to current master but the code has changed enough that I'm lost for how to translate the last of the original changes in this PR to the current api used in build.rs so I can test with the current version of blake3. Perhaps with some assistance I could get it over the line.

For context bevy recently started using blake3 for file hashing in the import pre-processing. We're unable to cross compile for now and investigating our options. Don't want you/project maintainers to feel pressured to attend to this though as there are backup alternatives :)

[oconnor663]

Ah, apologies for dropping the ball. I'm going to put a note on my calendar to get to this PR this weekend.

Incidentally I've just recently been studying up on how ECS works, and I've been looking at what Bevy does for atomically reserving ranges of IDs. So I'm pumped to see BLAKE3 getting involved :)

[oconnor663]

I'm trying to understand how this sort of cross-compilation works and how to test it. I'm getting some confusing results when I try an alternate approach with cross, where the master branch succeeds but this branch fails. Specifically following these instructions:

$ git clone https://github.com/cross-rs/cross
...
$ cd cross
$ git submodule update --init --remote
...
$ cargo build-docker-image x86_64-pc-windows-msvc-cross --tag local
... [suuuper long build]
$ cat > Cross.toml << EOF
[target.x86_64-pc-windows-msvc]
image = "ghcr.io/cross-rs/x86_64-pc-windows-msvc-cross:local"
EOF
$ cross test --target x86_64-pc-windows-msvc
...
test result: ok. 29 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s

If I rebase this PR and run the same tests, the last step fails:

$ cross test --target x86_64-pc-windows-msvc
...
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse2_
x86-64_windows_gnu.o" "-c" "c/blake3_sse2_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_sse2_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_sse2_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
  exit status: 0
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse41
_x86-64_windows_gnu.o" "-c" "c/blake3_sse41_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_sse41_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_sse41_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
  exit status: 0
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_avx2_
x86-64_windows_gnu.o" "-c" "c/blake3_avx2_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_avx2_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_avx2_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
...
  running: "lib.exe" "-out:/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/libblake3_sse2_sse41_avx2_assembly.a" "-nologo" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse2_x86-64_windows_gnu.o" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse41_x86-64_windows_gnu.o" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_avx2_x86-64_windows_gnu.o"
  \target\x86_64-pc-windows-msvc\debug\build\blake3-8bde2450d49dcdfd\out\c\blake3_sse2_x86-64_windows_gnu.o : fatal error LNK1107: invalid or corrupt file: cannot read at 0x4200
  exit status: 83

I'm guessing there's more than one way to cross-compile for the MSVC ABI, and whatever cross is doing there expects .asm files rather than .S files. Is that right? Is one of these approaches "more correct" than the other? Are there tweaks we could make to the build.rs changes here to make both approaches build?

[shrimpwtf]

Has this been fixed yet? Struggling to cross compile my binary because of this

[oconnor663]

Not yet, I need help answering the questions above.

[toothbrush7777777]

Hey @oconnor663

You may find this answer about differences between the MinGW and MSVC toolchains helpful.

Cross offers both toolchains. For the MSVC toolchain, Cross extracts headers and link libraries from the MSVC runtime packages, then runs the actual MSVC compiler under Wine emulation (both cl.exe and link.exe). Therefore, this PR would not work in this case since cl.exe wants the original MSVC assembly files.

You could get this PR to work using Cross, but you'd need to override the linker, C++ compiler and C compiler that Cross provides (the MSVC toolchain under Wine) to use instead the LLVM MSVC-compatible toolchain. Something like this should work:

[target.aarch64-unknown-linux-gnu]
pre-build = [
    # TODO: Install clang-cl, lld-link, etc.
    "export CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_LINKER=lld-link",
    # or maybe rust-lld would work?
    "export CC_x86_64_pc_windows_msvc=clang-cl",
    "export CXX_x86_64_pc_windows_msvc=clang-cl",
]

[toothbrush7777777]

It should be possible to "fix" the build script to work by default with MSVC on Windows, MSVC on Wine using Cross, or clang-cl on any architecture (e.g. using cargo xwin) by modifying the use_msvc_asm function. Something like this should work:

fn use_msvc_asm() -> bool {
    const MSVC_NAMES: &[&str] = &["", "cl", "cl.exe"];
    let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
    let target_env = env::var("CARGO_CFG_TARGET_ENV").unwrap_or_default();
    let target_windows_msvc = target_os == "windows" && target_env == "msvc";
    let host_triple = env::var("HOST").unwrap_or_default();
    let target_triple = env::var("TARGET").unwrap_or_default();
    let cross_compiling = host_triple != target_triple;
    let cc = env::var("CC").unwrap_or_default().to_ascii_lowercase();
    if !target_windows_msvc {
        // We are not building for Windows with the MSVC toolchain.
        false
    } else if !cross_compiling && MSVC_NAMES.contains(cc) {
        // We are building on Windows with the MSVC toolchain (and not cross-compiling for another architecture or target).
        true
    } else {
        // We are cross-compiling to Windows with the MSVC toolchain.
        let target_arch = env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_default();
        let target_vendor = env::var("CARGO_CFG_TARGET_VENDOR").unwrap_or_default();
        let cc = env::var(format!("CC_{target_arch}_{target_vendor}_windows_msvc")).unwrap_or_default().to_ascii_lowercase();
        // Check if we are using the MSVC compiler.
        MSVC_NAMES.contains(cc)
    }
}

Edit: This should support all the Windows targets now, including the UWP and new Windows 7 targets.

[VitalyAnkh]

Any updates? Does this PR need any assistance? If so, I'd be happy to help.

[casey]

Can't cross compile just because of this, unfortunately 😭

Edit: This also breaks not only cross compilation, but also even running cargo check --target x86_64-pc-windows-msvc on another platform.

[oconnor663]

@casey I've rebased part of this fix and included the change suggested by @toothbrush7777777. My branch is https://github.com/BLAKE3-team/BLAKE3/tree/fix-clang-cl-cross-rebase. When I clone just and override the blake3 dependency with path = ..., I find that that branch fixes cargo xwin check --target x86_64-pc-windows-msvc but not cargo check --target x86_64-pc-windows-msvc. Is that what you expected? Or maybe you have your environment set up differently, so cargo check --target x86_64-pc-windows-msvc works for you with that patch? (I get failed to find tool "ml64.exe") Could you let me know what you see?

[casey]

When I clone just and override the blake3 dependency with path = ..., I find that that branch fixes cargo xwin check --target x86_64-pc-windows-msvc.

Dope, just tested it and it works. I'm working on a really heinous feature for just which has different code on different platforms, so even just having a branch I can use for development to catch errors on Windows is super helpful. Thank you!

but not cargo check --target x86_64-pc-windows-msvc. Is that what you expect?

cargo check --target some-random-triple works for pure rust programs, for example, cargo check --target x86_64-pc-windows-msvc works on my mac for a hello world rust binary. But I don't know what the expectation is for anything with a nontrivial platform-specific build script.

Or maybe you have your environment set up differently, so cargo check --target x86_64-pc-windows-msvc works for you with that patch? (I get failed to find tool "ml64.exe") Could you let me know what you see?

cargo check --target x86_64-pc-windows-msvc fails with:

error occurred in cc-rs: failed to find tool "ml64.exe": No such file or directory (os error 2)

But that's probably normal for anything with a nontrivial build script.

[oconnor663]

I landed the build.rs changes from this PR (6da62fb) plus @toothbrush7777777's suggested change (0bdc6ca). The CI changes seemed to run into some trouble with multiple versions of clang-cl being installed(?), so I had better luck just using the upstream docker image provided by cargo-xwin (15e3b40). Using docker also incidentally worked around the problem where the AVX-512-not-supported warning was turned into an error, because docker doesn't pick up the BLAKE3_CI env var by default. I've confirmed that cherry-picking the new cargo xwin CI build onto a branch without this fix leads to a failure.