Skip to content

[release/6.0] x64 on ARM64 fixes #59295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Anipik merged 3 commits into from
Sep 20, 2021
Merged

[release/6.0] x64 on ARM64 fixes #59295

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
631b061
Retarget DOTNETHOME when installing x64 on ARM64 (#58669)
ericstj Sep 8, 2021
9be9002
Only set path in x64 mac installer when installing on x64 (#59210)
ericstj Sep 17, 2021
e1c5dc8
Updating dependencies from https://github.com/dotnet/arcade build 202…
ericstj Sep 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions eng/Version.Details.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>7324320f814152b72295946847ca72413507705a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Templating" Version="6.0.0-beta.21460.7">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>7324320f814152b72295946847ca72413507705a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Workloads" Version="6.0.0-beta.21460.7">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>7324320f814152b72295946847ca72413507705a</Sha>
Expand Down
1 change: 1 addition & 0 deletions eng/Versions.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
<MicrosoftDotNetBuildTasksArchivesVersion>6.0.0-beta.21460.7</MicrosoftDotNetBuildTasksArchivesVersion>
<MicrosoftDotNetBuildTasksInstallersVersion>6.0.0-beta.21460.7</MicrosoftDotNetBuildTasksInstallersVersion>
<MicrosoftDotNetBuildTasksPackagingVersion>6.0.0-beta.21460.7</MicrosoftDotNetBuildTasksPackagingVersion>
<MicrosoftDotNetBuildTasksTemplatingVersion>6.0.0-beta.21460.7</MicrosoftDotNetBuildTasksTemplatingVersion>
<MicrosoftDotNetBuildTasksWorkloadsPackageVersion>6.0.0-beta.21460.7</MicrosoftDotNetBuildTasksWorkloadsPackageVersion>
<MicrosoftDotNetRemoteExecutorVersion>6.0.0-beta.21460.7</MicrosoftDotNetRemoteExecutorVersion>
<MicrosoftDotNetVersionToolsTasksVersion>6.0.0-beta.21460.7</MicrosoftDotNetVersionToolsTasksVersion>
Expand Down
34 changes: 33 additions & 1 deletion src/installer/pkg/sfx/installers/dotnet-host.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<UseBrandingNameInLinuxPackageDescription>true</UseBrandingNameInLinuxPackageDescription>
<MacOSComponentNamePackType>sharedhost</MacOSComponentNamePackType>
<IncludeVersionInMacOSComponentName>false</IncludeVersionInMacOSComponentName>
<MacOSScriptsDirectory>osx_scripts/host</MacOSScriptsDirectory>
<MacOSScriptsTemplateDirectory>osx_scripts/host</MacOSScriptsTemplateDirectory>
<MacOSPackageDescription>The .NET Shared Host.</MacOSPackageDescription>
<RpmScriptsDirectory>$(MSBuildThisFileDirectory)rpm_scripts/host</RpmScriptsDirectory>
<RpmAfterInstallScript>$(RpmScriptsDirectory)/after_install.sh</RpmAfterInstallScript>
Expand Down Expand Up @@ -78,4 +78,36 @@
<RpmJsonProperty Include="@(PackageConflictsProperty)" />
</ItemGroup>
</Target>

<PropertyGroup>
<_MacOSIntermediatesPath>$(IntermediateOutputPath)macos/</_MacOSIntermediatesPath>
<!-- CreatePkg depends on this property being set.-->
<MacOSScriptsDirectory>$(_MacOSIntermediatesPath)scripts</MacOSScriptsDirectory>
</PropertyGroup>
<ItemGroup>
<_MacOSScript Include="$(MacOSScriptsTemplateDirectory)/*" Destination="$(MacOSScriptsDirectory)/%(FileName)%(Extension)"/>
</ItemGroup>
<Target Name="ReplaceTemplateParametersInMacOsScripts"
Inputs="@(_MacOSScript)"
Outputs="@(_MacOSScript->'%(Destination)')"
DependsOnTargets="_GetInstallerProperties"
BeforeTargets="CreatePkg">

<PropertyGroup>
<_UnameMachineRegex>$(InstallerTargetArchitecture)</_UnameMachineRegex>
<_UnameMachineRegex Condition="'$(InstallerTargetArchitecture)' == 'arm64'">arm64|aarch64</_UnameMachineRegex>
<_UnameMachineRegex Condition="'$(InstallerTargetArchitecture)' == 'x64'">amd64|x86_64</_UnameMachineRegex>
<_MacOSScriptsTemplateProperties>InstallerTargetArchitecture=$(InstallerTargetArchitecture);UnameMachineRegex=$(_UnameMachineRegex)</_MacOSScriptsTemplateProperties>
</PropertyGroup>

<GenerateFileFromTemplate TemplateFile="@(_MacOSScript)"
OutputPath="@(_MacOSScript->'%(Destination)')"
Properties="$(_MacOSScriptsTemplateProperties)">
<Output TaskParameter="ResolvedOutputPath" ItemName="FileWrites" />
</GenerateFileFromTemplate>
</Target>

<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Templating" Version="$(MicrosoftDotNetBuildTasksTemplatingVersion)" />
</ItemGroup>
</Project>
1 change: 0 additions & 1 deletion src/installer/pkg/sfx/installers/dotnet-hostfxr.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
<OutputFilesCandleVariable>HostFxrSrc</OutputFilesCandleVariable>
<UseBrandingNameInLinuxPackageDescription>true</UseBrandingNameInLinuxPackageDescription>
<MacOSComponentNamePackType>hostfxr</MacOSComponentNamePackType>
<MacOSScriptsDirectory>osx_scripts/hostfxr</MacOSScriptsDirectory>
<MacOSPackageDescription>The .NET HostFxr</MacOSPackageDescription>
</PropertyGroup>

Expand Down
37 changes: 31 additions & 6 deletions src/installer/pkg/sfx/installers/host.wxs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<Fragment>

<ComponentGroup Id="InstallSharedHostandDetectionKeys">
<ComponentGroup Id="InstallSharedHostandDetectionKeys" Directory="DOTNETHOME">

<!-- When installing the SharedHost; copy all files to the traditional default install location: 'ProgramFiles'\dotnet even when installing into a custom location.

Expand All @@ -20,17 +20,32 @@
1. the legacy SDK can be subsequently installed
2. the user runs 'dotnet' commands directly against 'ProgramFiles'\dotnet\dotnet.exe -->

<Component Id="cmpCoreHost" Directory="DOTNETHOME" Guid="{45399BBB-DDA5-4386-A2E9-618FB3C54A18}" >
<Component Id="cmpCoreHost" Guid="{45399BBB-DDA5-4386-A2E9-618FB3C54A18}" >
<File Id="fileCoreHostExe" KeyPath="yes" Source="$(var.HostSrc)\dotnet.exe">
<CopyFile Id="copyFileCoreHostExe" DestinationDirectory="PROGRAMFILES_DOTNET" />
</File>
</Component>

<Component Id="cmpSharedHostVersionRegistry" Guid="*">
<RegistryKey Root="HKLM" Key="SOFTWARE\dotnet\Setup\InstalledVersions\$(var.Platform)\sharedhost">
<RegistryValue Action="write" Name="Version" Type="string" Value="$(var.NugetVersion)"/>
<RegistryValue KeyPath="yes" Action="write" Name="Version" Type="string" Value="$(var.NugetVersion)"/>
</RegistryKey>
</Component>

<Component Id="cmdPath" Guid="*">
<?if $(var.Platform)~=x64 ?>
<!-- For x64 installer, only add to PATH when actually on native architecture -->
<Condition>NOT NON_NATIVE_ARCHITECTURE</Condition>
<?endif?>

<!-- A stable keypath with the right SxS characteristics for our PATH entry-->
<RegistryKey Root="HKLM" Key="SOFTWARE\dotnet\Setup\InstalledVersions\$(var.Platform)\sharedhost">
<RegistryValue KeyPath="yes" Action="write" Name="Path" Type="string" Value="[DOTNETHOME]"/>
</RegistryKey>
<Environment Id="E_PATH" Name="PATH" Value="[DOTNETHOME]" Part="last" Action="set" System="yes" />
</Component>

<Component Id="cmpLicenseFiles" Directory="DOTNETHOME" Guid="{A61CBE5B-1282-4F29-90AD-63597AA2372E}">
<Component Id="cmpLicenseFiles" Guid="{A61CBE5B-1282-4F29-90AD-63597AA2372E}">
<File Id="fileLicenseTxt" KeyPath="yes" Source="$(var.HostSrc)\LICENSE.txt">
<CopyFile Id="copyFileLicenseTxt" DestinationDirectory="PROGRAMFILES_DOTNET" />
</File>
Expand All @@ -47,6 +62,16 @@
<DirectoryRef Id="$(var.Program_Files)">
<Directory Id="PROGRAMFILES_DOTNET" Name="dotnet" />
</DirectoryRef>

<?if $(var.Platform)~=x64 ?>
<CustomActionRef Id="Set_PROGRAMFILES_DOTNET_NON_NATIVE_ARCHITECTURE" />
<?endif?>
</Fragment>

</Wix>

<Fragment>
<!-- Unlike DOTNETHOME which gives precedence to a user specified value over an x64 suffix, here we always want the suffixed path -->
<SetProperty Action="Set_PROGRAMFILES_DOTNET_NON_NATIVE_ARCHITECTURE" Id="PROGRAMFILES_DOTNET" Value="[$(var.Program_Files)]dotnet\x64\" After="Set_NON_NATIVE_ARCHITECTURE">
NON_NATIVE_ARCHITECTURE
</SetProperty>
</Fragment>
</Wix>
22 changes: 17 additions & 5 deletions src/installer/pkg/sfx/installers/osx_scripts/host/postinstall
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
#
# Licensed to the .NET Foundation under one or more agreements.
# The .NET Foundation licenses this file to you under the MIT license.
Expand All @@ -10,9 +10,21 @@ INSTALL_DESTINATION=$2
# A temporary fix for the permissions issue(s)
chmod 755 $INSTALL_DESTINATION/dotnet

# Add the installation directory to the system-wide paths
# But first create the directory if it doesn't exist
mkdir -p /etc/paths.d
echo $INSTALL_DESTINATION | tee /etc/paths.d/dotnet
if [ -e /etc/dotnet/install_location ]; then
# clear out any entries for this architecture if they exist
sed -i old '/^${InstallerTargetArchitecture}=/d' /etc/dotnet/install_location
else
mkdir -p /etc/dotnet
fi

echo ${InstallerTargetArchitecture}=$INSTALL_DESTINATION | tee -a /etc/dotnet/install_location

# if we're running on the native architecture
if [[ "$(uname -m)" =~ "${UnameMachineRegex}" ]]; then
Copy link
Member

@hoyosjs hoyosjs Sep 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uname -m will give you x86_64 under rosetta emulation. Is this the intended behavior?

Copy link
Member

@am11 am11 Sep 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone is really running this installer package under Rosetta emulation, then yes I think that should be the expected behavior.

Copy link
Member Author

@ericstj ericstj Sep 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a card to our board to test this. I'm not so worried about accidentally setting path, that's easy to fix and user can change it per their intent. I am worried if somehow we can lie to the installer javascript detection, since that would put the files in the wrong place. @sfoslund https://github.com/orgs/dotnet/projects/38#card-69051611

Copy link
Member

@hoyosjs hoyosjs Sep 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone is really running this installer package under Rosetta emulation, then yes I think that should be the expected behavior.

I believe this is against what was proposed - only setting path of dotnet for the native one. I'm ok with it either way.

Copy link
Member

@am11 am11 Sep 19, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is against what was proposed - only setting path of dotnet for the native one.

If user’s intention is to “Run as x86_64 app” on arm64 host, in that case “native” is the target of emulation/virtualization and x86_64 behavior is expected. This is how other software (including Apple’s own) handle this case.

Copy link
Member Author

@ericstj ericstj Sep 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I just tried this out and I think I found a way to test it.

If I create a terminal that runs using rosetta, then launch the installer from that ternminal I see it report hw.machine as x86_64 on my M1. This is not good, we don't want the installer to be lied to.

@sfoslund here's the command I ran to do this:
sudo installer -dumplog -verboseR -pkg testPkg.pkg -target /

We should check the newer detection you added and see if that's avoiding this redirection.

Copy link
Member Author

@ericstj ericstj Sep 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried the repro with latest detection and it also sees the x64 values when run from rosetta terminal.

I found at least one way to identify when this is happening: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment.

sysctl.proc_translated --> this returns 1 when run from rosetta. I wonder if that's a sufficient escape hatch for our purposes.

I opened #59370 to track this.

Copy link
Member

@am11 am11 Sep 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not good, we don't want the installer to be lied to.

I was considering it as a correct behavior based on how other installers (node.js, rustup and so on) operate in Rosetta2, and more generally, any emulation environments (qemu). Forcing our will against user wishes is no good, IMO.

Copy link
Member

@sfoslund sfoslund Sep 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my quick research, it looks like we can detect if we're running in rosetta with sysctl -in sysctl.proc_translated. Do we just want to try to block the install in this case? Or should we add this check to our logic for determining if a machine is x64?

Copy link
Member Author

@ericstj ericstj Sep 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@am11 perhaps you can open an issue with a scenario that you feel is blocked and we can discuss that?

We're actually making the runtime aware of emulation and making it so both x64 and ARM64 can coexist. The problem is that existing installers weren't aware and would clash. We're preserving the single entrypoint, so we do have to make the installers aware of real architecture so that they can avoid clashing. This is the part we can't have lied to, since that circumvents the work. The rest of the runtime is happy to be lied to, that's what the support for this is all about.

# Add the installation directory to the system-wide paths
# But first create the directory if it doesn't exist
mkdir -p /etc/paths.d
echo $INSTALL_DESTINATION | tee /etc/paths.d/dotnet
fi

exit 0
16 changes: 0 additions & 16 deletions src/installer/pkg/sfx/installers/osx_scripts/hostfxr/postinstall
View file
Open in desktop

This file was deleted.