[release/8.0] Move default build images to macos 15 (and xcode 16) (#120589) #122423

agocke · 2025-12-10T23:56:17Z

(cherry picked from commit eaafd7c) (cherry picked from commit 1905046)

Copilot

Pull request overview

This PR updates the default macOS build images from macOS 13 to macOS 15 (with Xcode 16) for public builds, and adds test exclusions for Swift interop stress tests that are incompatible with the new environment.

Updates public macOS build pool to use 'macos-15' image
Adds documentation comment explaining that build platforms should always use the latest available version
Excludes Swift ABI stress tests for mono minijit x64 configuration due to known issues on the new platform

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
eng/pipelines/common/xplat-setup.yml	Updates public macOS pool from 'macos-13' to 'macos-15' and adds policy comment about using latest platform versions
src/tests/issues.targets	Adds test exclusions for Swift interop stress tests (SwiftRetAbiStress and SwiftCallbackAbiStress) for mono minijit x64 configuration, referencing issue #121983

agocke · 2025-12-11T18:36:33Z

@steveisok it looks like mono build is failing. Any insight?

steveisok · 2025-12-11T19:02:05Z

@steveisok it looks like mono build is failing. Any insight?

Taking a look.

lambdageek

I guess this is cherrypicking from c48aa81

I don't remember the subtleties anymore, but this seems right

not quite there yet

lambdageek · 2025-12-11T22:17:05Z

@steveisok I think you also need c48aa81#diff-b2942ad0f35c702862be561fa03e56f885e6b362b4b5522a21e23cc4d9e444ba the aot-compiler.c changes

lambdageek · 2025-12-11T23:32:03Z

On main this line

runtime/src/mono/mono/mini/decompose.c

Line 221 in ac34532

MONO_EMIT_NEW_LCOMPARE_IMM (cfg, ins->sreg1, (-(int)2147483648));

has INT_MIN

runtime/src/mono/mono/mini/decompose.c

Line 220 in 30a5c7c

MONO_EMIT_NEW_LCOMPARE_IMM (cfg, ins->sreg1, INT_MIN);

fixed here:

c85492f

steveisok · 2025-12-11T23:37:53Z

fixed here:

c85492f

Thanks!

dotnet-policy-service · 2025-12-12T09:35:14Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

agocke · 2026-01-07T19:08:41Z

From David: this smells like some sort of corruption, not debuggable from a trace. We'll need a dump and probably access to one of the Helix machines + a live repro.

janvorli · 2026-01-07T21:44:36Z

@janvorli does this stack trace look familiar?

@agocke no, it does not. Can you please share a link to the log file from which you've copied that trace? I can try to repro it locally.

janvorli · 2026-01-09T01:47:24Z

I can repro the issue locally executing the bits from the failed run pulled by runfo. And yet it works perfectly fine if I build it locally using exactly the same XCode tools version, repeating exactly the command lines that CI uses to build the stuff on my arm64 mac running the whole build under rosetta so that it matches build on x64 mac.
So it seems to indicate something is wrong with the build machines.
It is hard to reason about the problem using the build from the CI as the libcoreclr.dylib is missing symbols and there is no libcoreclr.dylib.dwarf file.

janvorli · 2026-01-09T12:34:18Z

Update: I can repro it with local build now, I haven't realized libraries tests on macOS use release build of the runtime.

Co-authored-by: Eric StJohn <[email protected]> Co-authored-by: Matous Kozak <[email protected]> (cherry picked from commit eaafd7c) (cherry picked from commit 1905046)

…and mess things up.

…ncorrect transformations by the ld64/ld-prime linkers

steveisok · 2026-01-10T12:12:41Z

Looks like there is a consistent crash in System.Net.Http.Functional.Tests

steveisok · 2026-01-10T18:39:50Z

I believe we are running into #97966 as a result of the bump.

@dotnet/ncl can you please confirm? I'd like to move this forward as 8.0 servicing is blocked until we can get this PR in.

karelz · 2026-01-12T14:50:13Z

@steveisok do you have details about the reliable crash? @rzikm @wfurt @liveans should be able to help ...

steveisok · 2026-01-12T15:54:00Z

@steveisok do you have details about the reliable crash? @rzikm @wfurt @liveans should be able to help ...

I didn't try to get a managed stack yet, but I think the native one gives a pretty good indicator it's the issue I linked to.

Thread 51 Crashed:: .NET TP Worker
0   libsystem_kernel.dylib        	    0x7ff811555846 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff811590b66 pthread_kill + 259
2   libsystem_c.dylib             	    0x7ff8114af686 __abort + 145
3   libsystem_c.dylib             	    0x7ff8114af5f5 abort + 141
4   libsystem_malloc.dylib        	    0x7ff8113a6ae8 malloc_vreport + 857
5   libsystem_malloc.dylib        	    0x7ff8113aa589 malloc_report + 151
6   GSS                           	    0x7ff8240fce40 _gss_scram_release_cred + 96
7   GSS                           	    0x7ff8240fb873 _gss_scram_delete_sec_context + 83
8   GSS                           	    0x7ff8240c6a36 gss_delete_sec_context + 166
9   GSS                           	    0x7ff8240cc58d _gss_spnego_internal_delete_sec_context + 237
10  GSS                           	    0x7ff8240cc487 _gss_spnego_delete_sec_context + 87
11  GSS                           	    0x7ff8240c6a36 gss_delete_sec_context + 166
12  libSystem.Net.Security.Native.dylib	       0x1a8e23371 NetSecurityNative_DeleteSecContext + 145 (pal_gssapi.c:190)
13  ???                           	       0x109711430 ???

wfurt · 2026-01-12T17:43:06Z

gssapi was buggy on MacOS and we switched to managed implementation for NTLM. But I think we still use it for Kerberos/Negotiate. It seems like the new MacOS has more problems and that will need to be investigated.

wfurt · 2026-01-12T17:45:41Z

ah, it seems like the comment says it was resolved in .NET 9+ #97966
I'm not sure if the managed implementation is complete in 8.0 and ready to be used.

steveisok · 2026-01-12T18:07:47Z

ah, it seems like the comment says it was resolved in .NET 9+ #97966 I'm not sure if the managed implementation is complete in 8.0 and ready to be used.

Do you think it's related to gss and not a runtime issue? If so, I think we can move this forward and decide what to do separately.

agocke · 2026-01-12T18:08:39Z

I think taking this PR, even with System.Net.Http failing, is better than the alternative. We currently have no working Mac builds. At worst we get a Mac build with a bug in one area.

/ba-g Known issue in HTTP tests and dead letter in other queues

steveisok · 2026-01-12T18:10:04Z

I think taking this PR, even with System.Net.Http failing, is better than the alternative. We currently have no working Mac builds. At worst we get a Mac build with a bug in one area.

/ba-g Known issue in HTTP tests and dead letter in other queues

I agree. No complaints from me :-)

Copilot AI review requested due to automatic review settings December 10, 2025 23:56

github-actions bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Dec 10, 2025

Copilot started reviewing on behalf of agocke December 10, 2025 23:57 View session

Copilot AI reviewed Dec 10, 2025

View reviewed changes

dotnet-policy-service bot assigned agocke Dec 11, 2025

This was referenced Dec 11, 2025

Assertion failed in SafeFileHandle.OverlappedValueTaskSource.IOCallback #99517

Open

wasm build failure in CI #117017

Open

agocke changed the title ~~Move default build images to macos 15 (and xcode 16) (#120589)~~ [release/8.0] Move default build images to macos 15 (and xcode 16) (#120589) Dec 11, 2025

agocke added the Servicing-approved Approved for servicing release label Dec 11, 2025

steveisok requested review from lambdageek and thaystg as code owners December 11, 2025 20:26

steveisok requested review from lewing and removed request for lambdageek December 11, 2025 20:26

lambdageek previously approved these changes Dec 11, 2025

View reviewed changes

steveisok requested a review from SamMonoRT as a code owner December 11, 2025 22:50

This was referenced Dec 12, 2025

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

[android-arm64] The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#6408

Open

akoeplinger added area-Infrastructure and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Dec 12, 2025

github-project-automation bot added this to Runtime Infra Dec 12, 2025

build-analysis bot mentioned this pull request Jan 7, 2026

[8.0] Long running test timeout: Wasm.Build.Tests.TestAppScenarios.WasmAppBuilderDebugLevelTests.BuildWithExplicitValue #114698

Open

agocke and others added 7 commits January 9, 2026 15:52

Move default build images to macos 15 (and xcode 16) (dotnet#120589)

32cb9b5

Co-authored-by: Eric StJohn <[email protected]> Co-authored-by: Matous Kozak <[email protected]> (cherry picked from commit eaafd7c) (cherry picked from commit 1905046)

Replace the use of in some macros in Mono - it will expand to in C11 …

7d9e6da

…and mess things up.

One more

e3a45c7

Fix warning on resolved in dotnet#98018

a57b9a8

Also bump Helix testing to macos 14

a444b21

Upgrade libraries queues

dbf26e8

Mark labels in x64 write barrier code as local/alt_entry to prevent i…

b1df24b

…ncorrect transformations by the ld64/ld-prime linkers

agocke force-pushed the 8.0-macos-build-upgrade branch from 93ef2d6 to b1df24b Compare January 9, 2026 23:52

build-analysis bot mentioned this pull request Jan 10, 2026

[Test][iOS] iOS.Device.Aot.Test fails with APP_LAUNCH_FAILURE #119665

Open

Bump the internal pool to use macos-15 too.

e21e854

agocke merged commit 8e8c0c7 into dotnet:release/8.0-staging Jan 12, 2026
182 of 190 checks passed

agocke deleted the 8.0-macos-build-upgrade branch January 12, 2026 18:09

github-project-automation bot moved this to Done in Runtime Infra Jan 12, 2026

jeffhandley mentioned this pull request Jan 13, 2026

SIGABRT in System.Net.Http.Functional.Tests on macOS 14 #123157

Open

MihuBot mentioned this pull request Jan 13, 2026

[✨ Triage] dotnet/runtime#123157 by jeffhandley - SIGABRT in System.Net.Http.Functional.Tests on macOS 14 MihuBot/dotnet-triage#74

Open

steveisok mentioned this pull request Jan 13, 2026

[manual] Merge release/8.0-staging into release/8.0 #123114

Merged

[release/8.0] Move default build images to macos 15 (and xcode 16) (#120589) #122423

[release/8.0] Move default build images to macos 15 (and xcode 16) (#120589) #122423

Uh oh!

Conversation

agocke commented Dec 10, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

agocke commented Dec 11, 2025

Uh oh!

steveisok commented Dec 11, 2025

Uh oh!

lambdageek left a comment

Choose a reason for hiding this comment

Uh oh!

lambdageek commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lambdageek commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveisok commented Dec 11, 2025

Uh oh!

dotnet-policy-service bot commented Dec 12, 2025

Uh oh!

agocke commented Jan 7, 2026

Uh oh!

janvorli commented Jan 7, 2026

Uh oh!

janvorli commented Jan 9, 2026

Uh oh!

janvorli commented Jan 9, 2026

Uh oh!

steveisok commented Jan 10, 2026

Uh oh!

steveisok commented Jan 10, 2026

Uh oh!

karelz commented Jan 12, 2026

Uh oh!

steveisok commented Jan 12, 2026

Uh oh!

wfurt commented Jan 12, 2026

Uh oh!

wfurt commented Jan 12, 2026

Uh oh!

steveisok commented Jan 12, 2026

Uh oh!

agocke commented Jan 12, 2026

Uh oh!

Uh oh!

steveisok commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

lambdageek commented Dec 11, 2025 •

edited

Loading

lambdageek commented Dec 11, 2025 •

edited

Loading