Skip to content

Conversation

@kotlarmilos
Copy link
Member

@kotlarmilos kotlarmilos commented Nov 3, 2025

Description

This PR updates the MAUI iOS samples to use .NET 11 workloads and adds support for running with the CoreCLR interpreter.

Changes

Validation

Test run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2862740&view=results

Out of scope

Device startup measurements are currently unreliable and may fail due to a device connection issue. This is a known issue and is being handled separately by the engineering team.

Contributes to dotnet/runtime#120054

Fixes #5054

@kotlarmilos kotlarmilos self-assigned this Nov 3, 2025
@kotlarmilos kotlarmilos added the enhancement New feature or request label Nov 3, 2025
@kotlarmilos kotlarmilos changed the title Add Maui iOS CoreCLR scenario benchmarks to the performance job pipeline [WIP] Add Maui iOS CoreCLR scenario benchmarks to the performance job pipeline Nov 3, 2025
@kotlarmilos kotlarmilos marked this pull request as ready for review November 28, 2025 15:08
@kotlarmilos kotlarmilos changed the title [WIP] Add Maui iOS CoreCLR scenario benchmarks to the performance job pipeline Add Maui iOS CoreCLR scenario benchmarks to the performance job pipeline Nov 28, 2025
@kotlarmilos kotlarmilos marked this pull request as draft December 10, 2025 12:55
@kotlarmilos kotlarmilos changed the title Add Maui iOS CoreCLR scenario benchmarks to the performance job pipeline Update MAUI iOS samples to .NET 11 workloads and add CoreCLR interpreter support Dec 16, 2025
@kotlarmilos
Copy link
Member Author

kotlarmilos commented Dec 16, 2025

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates MAUI iOS samples to use .NET 11 workloads and adds support for running with the CoreCLR interpreter. The changes temporarily disable certain MAUI scenarios until .NET 11 packages are available, update SDK and workload package versions, and introduce CoreCLR interpreter configuration options.

  • Removes the generic --nativeaot command-line argument in favor of MSBuild properties for more granular runtime and codegen control
  • Adds CoreCLR interpreter support for iOS benchmarks alongside existing Mono and NativeAOT configurations
  • Updates macOS/iOS SDK dependencies from version 18.5 to 26.1 with .NET 11 compatibility

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/scenarios/shared/precommands.py Removes deprecated --nativeaot argument and related command-line handling
src/scenarios/netios/pre.py Adds /p:TargetsCurrent=true workaround for .NET 11 workload availability
src/scenarios/mauiios/pre.py Adds /p:TargetsCurrent=true workaround with indentation issue in comment
src/scenarios/mauiblazorios/pre.py Adds /p:TargetsCurrent=true workaround for Blazor template
scripts/run_performance_job.py Adds --break-system-packages flag for pip on macOS to handle system Python restrictions
scripts/dotnet.py Generalizes SDK version detection to support major version increments (e.g., net10.0 → SDK 11.0)
scripts/channel_map.py Adds nativeaot11.0 channel mapping for .NET 11 NativeAOT builds
eng/pipelines/sdk-perf-jobs.yml Adds new CoreCLR interpreter job configuration for MAUI iOS scenarios
eng/performance/maui_scenarios_ios.proj Replaces --nativeaot with MSBuild properties, updates Xcode version, temporarily disables MAUI templates, contains typo in commented test command
eng/Version.Details.xml Updates macOS/iOS SDK dependencies from 18.5 to 26.1 versions with new commit SHA

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "--upgrade", "pip"]).run()
RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "urllib3==1.26.19"]).run()
RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "requests"]).run()
break_system_packages = ["--break-system-packages"] if args.os_group == "osx" else []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To double check, the new --break-system-packages will hopefully allow these calls to succeed on OSX because without it we get an error about the environment being externally managed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it fixes the failures in #5054

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we don't create a python environment (venv) and instead we use the global system python? cc: @LoopedBard3

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we should be using a venv looking at the logs from the runs, so maybe there is just a different interaction on OSX that causes the error to still show up despite being in a venv. Here is part of the installing prereqs near the top of job from the test run:

+ export ORIGPYPATH=:/etc/helix/scripts
+ ORIGPYPATH=:/etc/helix/scripts
+ echo '** Installing prerequistes **'
** Installing prerequistes **
+ python3 -m venv /tmp/helix/working/B7720975/w/9D7D093E/e/.venv
+ . /tmp/helix/working/B7720975/w/9D7D093E/e/.venv/bin/activate
++ deactivate nondestructive
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/tmp/helix/working/B7720975/w/9D7D093E/e/.venv
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/bin:/bin:/usr/sbin:/sbin
++ PATH=/tmp/helix/working/B7720975/w/9D7D093E/e/.venv/bin:/usr/bin:/bin:/usr/sbin:/sbin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ PS1='(.venv) '
++ export PS1
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
+ export PYTHONPATH=
+ PYTHONPATH=
+ python -m pip install -U pip
Requirement already satisfied: pip in ./.venv/lib/python3.9/site-packages (21.2.4)
Collecting pip
  Using cached pip-25.3-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-25.3
+ python -m pip install azure.storage.blob==12.13.0
Collecting azure.storage.blob==12.13.0
  Using cached azure_storage_blob-12.13.0-py3-none-any.whl.metadata (25 kB)
Collecting azure-core<2.0.0,>=1.23.1 (from azure.storage.blob==12.13.0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Parker. Looking at the log, this seems to be coming from the helix machine but the commands above are run on the macOS build image where I don't see .venv being activated https://dev.azure.com/dnceng/internal/_build/results?buildId=2862740&view=logs&j=007c6ee1-d4e8-565e-80d2-5b93cbca4c6a&t=e8845911-7767-5da3-8d86-0ee0806c6d02

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. I am not aware of any reason we are not activating the .venv there other than us not having added that to the flow. @DrewScoggins anything to add to that?

matouskozak
matouskozak previously approved these changes Dec 17, 2025
RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "--upgrade", "pip"]).run()
RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "urllib3==1.26.19"]).run()
RunCommand([*(agent_python.split(" ")), "-m", "pip", "install", "--user", "requests"]).run()
break_system_packages = ["--break-system-packages"] if args.os_group == "osx" else []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we don't create a python environment (venv) and instead we use the global system python? cc: @LoopedBard3

LoopedBard3
LoopedBard3 previously approved these changes Dec 17, 2025
Copy link
Member

@LoopedBard3 LoopedBard3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small question about spacing in mauiios pre.py, but looks good otherwise.

Copy link
Member

@LoopedBard3 LoopedBard3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MAUI iOS scenarios fail to run performance measurements

4 participants