Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
e0c4a60
Enable AddressSanitizer in CoreCLR, Libs, and Host and update runtime…
jkoritzinsky Mar 16, 2023
6fc2555
Unify on HAS_ADDRESS_SANITIZER define name
jkoritzinsky Mar 17, 2023
d7fe870
Remove CRT runtime changes. We won't need this when we finally onboar…
jkoritzinsky Mar 24, 2023
007da09
Remove suppression now that we've fixed the underlying issue.
jkoritzinsky Mar 27, 2023
8d3615d
Remove some Windows-only sanitizer CMake that we won't need when ASAN…
jkoritzinsky Mar 27, 2023
92c4858
Set schedule for runtime-sanitized pipeline
jkoritzinsky Mar 27, 2023
a1f9900
Remove workaround in JIT memory set/copy helpers now that the JIT bug…
jkoritzinsky Apr 4, 2023
2978de4
Add missing helix queues setup template.
jkoritzinsky Apr 5, 2023
203e70c
Fix missing command to build nativeaot runtime tests as nativeaot.
jkoritzinsky Apr 7, 2023
eb9bf22
Add a scenario name to trigger the extended timeout in the libraries …
jkoritzinsky Apr 7, 2023
413410b
Remove extraneous whitespace.
jkoritzinsky Apr 7, 2023
da30efe
Fix using the cross-targetting ILC when sanitizers are enabled. Also …
jkoritzinsky Apr 7, 2023
b99fab9
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Apr 7, 2023
cd167b3
Disable LSAN on the CustomMain test.
jkoritzinsky Apr 10, 2023
8fcc66f
Disable some tests on sanitized runtimes.
jkoritzinsky Apr 10, 2023
931f444
Copy the sanitizer runtime for OSX NativeAOT runtime tests.
jkoritzinsky Apr 10, 2023
6732866
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Apr 17, 2023
3c7dd54
A little cleanup to try to get the build right now that we are always…
jkoritzinsky Apr 17, 2023
04d2d9f
Fix crossgen-corelib.proj syntax.
jkoritzinsky Apr 18, 2023
97c848f
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Apr 18, 2023
9946108
Fix cross-os dac builds to not include the host architecture in the o…
jkoritzinsky Apr 19, 2023
982e701
Split the debugger components into a separate component and make the …
jkoritzinsky Apr 20, 2023
6239952
Merge branch 'asan' of /mnt/e/source/runtime into asan
jkoritzinsky Apr 20, 2023
6769613
Always write out the host-arch path on Windows and update the cross-d…
jkoritzinsky Apr 21, 2023
35d9bd7
Change to use the dynamic runtime on Windows, as ASAN in VS is moving…
jkoritzinsky Apr 21, 2023
8b19fd3
Merge branch 'asan' of github.com:jkoritzinsky/runtime into asan
jkoritzinsky Apr 21, 2023
5ae90d3
Update docker images to include the sanitizer runtimes in the crossro…
jkoritzinsky Apr 21, 2023
647f73b
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Apr 21, 2023
5873281
Fix explicit image tags
jkoritzinsky Apr 21, 2023
d996328
The unsanitized build should be of the target architecture, not the h…
jkoritzinsky Apr 24, 2023
2e452d4
Merge branch 'main' into asan
jkoritzinsky Apr 25, 2023
2130d7b
Merge branch 'asan' of github.com:jkoritzinsky/runtime into asan
jkoritzinsky Apr 25, 2023
dc26f51
Turn off using the sigaltstack for NativeAOT tests that don't use the…
jkoritzinsky Apr 25, 2023
6cc6ad8
Make sure the shared ASAN runtime is present for the nativeaot/SmokeT…
jkoritzinsky Apr 27, 2023
591093e
Disable crossgen2 tests with sanitizers as they don't get us interest…
jkoritzinsky Apr 28, 2023
d285048
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Apr 28, 2023
7187ff2
Fix custom default options and disable some more crossgen2-based tests.
jkoritzinsky May 1, 2023
6448d19
Simplify lookup of asan runtime on mac and fix copying the shared run…
jkoritzinsky May 3, 2023
113f5b2
Merge branch 'asan' of github.com:jkoritzinsky/runtime into asan
jkoritzinsky May 3, 2023
11cccdf
Disable test that's failing for weird reasons.
jkoritzinsky May 4, 2023
c0357af
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky May 4, 2023
133e99b
Fix one more alloc-dealloc mismatch that only started to show up afte…
jkoritzinsky May 4, 2023
b863d6f
Disable the System.Text.Json test suite on sanitized builds as it cau…
jkoritzinsky May 5, 2023
8a51e72
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky May 5, 2023
98ed710
Fix test exclusion
jkoritzinsky May 8, 2023
4efdf96
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky May 15, 2023
6855c78
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky May 19, 2023
db5b069
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Jun 7, 2023
6383866
PR feedback.
jkoritzinsky Jun 7, 2023
0f0fdf5
Merge remote-tracking branch 'dotnet/main' into asan
jkoritzinsky Jun 12, 2023
36a450e
Fix mac build
jkoritzinsky Jun 12, 2023
f6987ed
Merge branch 'main' into asan
jkoritzinsky Jul 13, 2023
619966b
Do review changes
jkoritzinsky Jul 14, 2023
929a042
Add libbootstrapper object files to the platform manifest now that Na…
jkoritzinsky Jul 14, 2023
5913ca4
Disable tests that check size on sanitized builds
jkoritzinsky Jul 14, 2023
b8b3762
Use the built-in `include_guard` option
jkoritzinsky Jul 14, 2023
c2433e5
Merge branch 'main' of github.com:dotnet/runtime into asan
jkoritzinsky Jul 14, 2023
fa15f17
Disable use-after-return checking in ASAN. CoreCLR doesn't do well wi…
jkoritzinsky Jul 17, 2023
757a329
Only pass the no UAR flag on C and CXX with Clang (not AppleClang).
jkoritzinsky Jul 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions eng/Subsets.props
Original file line number Diff line number Diff line change
Expand Up @@ -278,17 +278,17 @@
Category="clr" />
</ItemGroup>

<!-- Build the cross-arch tools of CoreCLR for the same target architecture as an unsanitized build whenever we build a sanitized coreclr build. -->
<ItemGroup Condition="'$(ClrRuntimeBuildSubsets)' != '' and '$(CrossBuild)' != 'true' and '$(EnableNativeSanitizers)' != ''">
<!-- Build the debugging components of CoreCLR for the same target architecture as an unsanitized build whenever we build a sanitized coreclr build. -->
<ItemGroup Condition="'$(ClrRuntimeBuildSubsets)' != '' and '$(EnableNativeSanitizers)' != ''">
<ProjectToBuild
Include="$(CoreClrProjectRoot)runtime.proj"
AdditionalProperties="%(AdditionalProperties);
ClrCrossComponentsSubset=true;
ClrDebugSubset=true;
HostArchitecture=$(BuildArchitecture);
PgoInstrument=false;
NoPgoOptimize=true;
CrossBuild=false;
CMakeArgs=$(CMakeArgs) -DCLR_CROSS_COMPONENTS_BUILD=1"
CrossBuild=$(CrossBuild);
BuildSubdirectory=unsanitized"
UndefineProperties="EnableNativeSanitizers"
Category="clr" />
</ItemGroup>
Expand All @@ -311,7 +311,7 @@
PgoInstrument=false;
NoPgoOptimize=true;
TargetOS=linux;
CMakeArgs=$(CMakeArgs) -DCLR_CROSS_COMPONENTS_BUILD=1" Category="clr" />
CMakeArgs=$(CMakeArgs) -DCLR_CROSS_COMPONENTS_BUILD=1 -DBUILD_DEBUGGER_CROSS_COMPONENTS=1" Category="clr" />
</ItemGroup>

<ItemGroup Condition="$(_subset.Contains('+alpinedac+')) and $([MSBuild]::IsOsPlatform(Windows)) and '$(TargetArchitecture)' != 'x86'">
Expand All @@ -323,7 +323,7 @@
PgoInstrument=false;
NoPgoOptimize=true;
TargetOS=alpine;
CMakeArgs=$(CMakeArgs) -DCLR_CROSS_COMPONENTS_BUILD=1" Category="clr" />
CMakeArgs=$(CMakeArgs) -DCLR_CROSS_COMPONENTS_BUILD=1 -DBUILD_DEBUGGER_CROSS_COMPONENTS=1" Category="clr" />
</ItemGroup>

<ItemGroup Condition="$(_subset.Contains('+crossdacpack+'))">
Expand Down
16 changes: 14 additions & 2 deletions src/coreclr/build-runtime.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ set __Ninja=1
set __RequestedBuildComponents=
set __OutputRid=
set __ExplicitHostArch=
set __SubDir=

:Arg_Loop
if "%1" == "" goto ArgsDone
Expand Down Expand Up @@ -130,6 +131,7 @@ if [!__PassThroughArgs!]==[] (
if /i "%1" == "-hostarch" (set __HostArch=%2&set __ExplicitHostArch=1&shift&shift&goto Arg_Loop)
if /i "%1" == "-os" (set __TargetOS=%2&shift&shift&goto Arg_Loop)
if /i "%1" == "-outputrid" (set __OutputRid=%2&shift&shift&goto Arg_Loop)
if /i "%1" == "-subdir" (set __SubDir=%2&shift&shift&goto Arg_Loop)

if /i "%1" == "-cmakeargs" (set __CMakeArgs=%2 %__CMakeArgs%&set __remainingArgs="!__remainingArgs:*%2=!"&shift&shift&goto Arg_Loop)
if /i "%1" == "-configureonly" (set __ConfigureOnly=1&set __BuildNative=1&shift&goto Arg_Loop)
Expand Down Expand Up @@ -217,8 +219,15 @@ set "__ArtifactsIntermediatesDir=%__RepoRootDir%\artifacts\obj\coreclr\"
if "%__Ninja%"=="0" (set "__IntermediatesDir=%__IntermediatesDir%\ide")
set "__PackagesBinDir=%__BinDir%\.nuget"

if "%__ExplicitHostArch%" == "1" set __BinDir=%__BinDir%\%__HostArch%
if "%__ExplicitHostArch%" == "1" set __IntermediatesDir=%__IntermediatesDir%\%__HostArch%
if "%__ExplicitHostArch%" == "1" (
set __BinDir=%__BinDir%\%__HostArch%
set __IntermediatesDir=%__IntermediatesDir%\%__HostArch%
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this just added for parity with the linux scripts? If you haven't already, it'd be worth checking that it doesn't introduce unwanted changes to the output dir in our windows build scenarios.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need this for sanitized windows builds/tests. A version of ASAN with the fixes we need isn't out yet (which is why there isn't a Windows test leg in this PR) but I've been testing locally with the fixes.


if NOT "%__SubDir%"=="" (
set __BinDir=%__BinDir%\%__SubDir%
set __IntermediatesDir=%__IntermediatesDir%\%__SubDir%
)

REM Generate path to be set for CMAKE_INSTALL_PREFIX to contain forward slash
set "__CMakeBinDir=%__BinDir%"
Expand Down Expand Up @@ -318,6 +327,9 @@ for /f "delims=" %%a in ("-%__RequestedBuildComponents%-") do (
if not "!string:-crosscomponents-=!"=="!string!" (
set __CMakeTarget=!__CMakeTarget! crosscomponents
)
if not "!string:-debug-=!"=="!string!" (
set __CMakeTarget=!__CMakeTarget! debug
)
)
if "!__CMakeTarget!" == "" (
set __CMakeTarget=install
Expand Down
13 changes: 12 additions & 1 deletion src/coreclr/build-runtime.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ usage_list+=("-pgoinstrument: generate instrumented code for profile guided opti
usage_list+=("-skipcrossarchnative: Skip building cross-architecture native binaries.")
usage_list+=("-staticanalyzer: use scan_build static analyzer.")
usage_list+=("-component: Build individual components instead of the full project. Available options are 'hosts', 'jit', 'runtime', 'paltests', 'alljits', 'iltools', 'nativeaot', and 'spmi'. Can be specified multiple times.")
usage_list+=("-forcearchdir: Append a directory with the name of the host architecture to the intermediate and output paths.")
usage_list+=("-subdir: Append a directory with the provided name to the obj and bin paths.")

setup_dirs_local()
{
Expand Down Expand Up @@ -54,6 +54,11 @@ handle_arguments_local() {
__RequestedBuildComponents="$__RequestedBuildComponents $2"
__ShiftArgs=1
;;

subdir|-subdir)
__SubDir="$2"
__ShiftArgs=1
;;

*)
__UnprocessedBuildArgs="$__UnprocessedBuildArgs $1"
Expand Down Expand Up @@ -100,6 +105,7 @@ __UseNinja=0
__VerboseBuild=0
__CMakeArgs=""
__RequestedBuildComponents=""
__SubDir=""

source "$__ProjectRoot"/_build-commons.sh

Expand All @@ -121,6 +127,11 @@ if [[ "$__ExplicitHostArch" == 1 ]]; then
__BinDir="$__BinDir/$__HostArch"
fi

if [[ -n "$__SubDir" ]]; then
__IntermediatesDir="$__IntermediatesDir/$__SubDir"
__BinDir="$__BinDir/$__SubDir"
fi

# CI_SPECIFIC - On CI machines, $HOME may not be set. In such a case, create a subfolder and set the variable to set.
# This is needed by CLI to function.
if [[ -z "$HOME" ]]; then
Expand Down
4 changes: 4 additions & 0 deletions src/coreclr/components.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ add_component(paltests paltests_install)
add_component(iltools)
add_component(nativeaot)
add_component(spmi)
add_component(debug)

# Define coreclr_all as the fallback component and make every component depend on this component.
# iltools and paltests should be minimal subsets, so don't add a dependency on coreclr_misc
Expand All @@ -19,6 +20,9 @@ add_dependencies(runtime coreclr_misc)
# The runtime build requires the clrjit and iltools builds
add_dependencies(runtime jit iltools)

# The runtime build requires the debugger tools builds
add_dependencies(runtime debug)

add_dependencies(runtime hosts)

# The cross-components build is separate, so we don't need to add a dependency on coreclr_misc
Expand Down
22 changes: 8 additions & 14 deletions src/coreclr/crosscomponents.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -6,40 +6,34 @@ if (CLR_CMAKE_HOST_OS STREQUAL CLR_CMAKE_TARGET_OS)
COMPONENT crosscomponents
)

if (CLR_CMAKE_TARGET_ARCH_ARM OR CLR_CMAKE_TARGET_ARCH_ARM64)
if (CLR_CMAKE_TARGET_OSX AND ARCH_TARGET_NAME STREQUAL arm64)
install_clr (TARGETS
clrjit_universal_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
DESTINATIONS .
COMPONENT crosscomponents
)
elseif (CLR_CMAKE_TARGET_UNIX)
elseif (CLR_CMAKE_TARGET_ARCH_ARM OR CLR_CMAKE_TARGET_ARCH_ARM64)
install_clr (TARGETS
clrjit_unix_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
clrjit_universal_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
DESTINATIONS .
COMPONENT crosscomponents
)
else()
elseif (CLR_CMAKE_TARGET_UNIX)
install_clr (TARGETS
clrjit_win_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
clrjit_unix_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
DESTINATIONS .
COMPONENT crosscomponents
)
endif()

if (CLR_CMAKE_HOST_ARCH STREQUAL CLR_CMAKE_TARGET_ARCH)
else()
install_clr (TARGETS
mscordaccore
mscordbi
clrjit_win_${ARCH_TARGET_NAME}_${ARCH_HOST_NAME}
DESTINATIONS .
COMPONENT crosscomponents
)
endif()
endif()

# We support the DAC as a cross-component for a few specific cases:
# - Same OS and architecture, sanitized runtime with unsanitized DAC
# - Windows Host OS, any target OS, different architectures with the same bitness
if((NOT CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_HOST_APPLE AND NOT FEATURE_CROSSBITNESS) OR (CLR_CMAKE_HOST_OS STREQUAL CLR_CMAKE_TARGET_OS AND CLR_CMAKE_HOST_ARCH STREQUAL CLR_CMAKE_TARGET_ARCH))
if(NOT CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_HOST_APPLE AND NOT FEATURE_CROSSBITNESS)
install_clr (TARGETS
mscordaccore
mscordbi
Expand Down
2 changes: 1 addition & 1 deletion src/coreclr/dlls/mscordac/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ endif(CLR_CMAKE_HOST_UNIX)
target_link_libraries(mscordaccore PRIVATE ${COREDAC_LIBRARIES})

# add the install targets
install_clr(TARGETS mscordaccore DESTINATIONS . sharedFramework COMPONENT runtime)
install_clr(TARGETS mscordaccore DESTINATIONS . sharedFramework COMPONENT debug)

if(CLR_CMAKE_HOST_WIN32)
set(LONG_NAME_HOST_ARCH ${CLR_CMAKE_HOST_ARCH})
Expand Down
2 changes: 1 addition & 1 deletion src/coreclr/dlls/mscordbi/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -126,4 +126,4 @@ elseif(CLR_CMAKE_HOST_UNIX)
endif(CLR_CMAKE_HOST_WIN32)

# add the install targets
install_clr(TARGETS mscordbi DESTINATIONS . sharedFramework COMPONENT runtime)
install_clr(TARGETS mscordbi DESTINATIONS . sharedFramework COMPONENT debug)
2 changes: 2 additions & 0 deletions src/coreclr/runtime.proj
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
<_CoreClrBuildArg Condition="'$(HostArchitecture)' != ''" Include="-hostarch $(HostArchitecture)" />
<_CoreClrBuildArg Condition="'$(EnableNativeSanitizers)' != ''" Include="-fsanitize $(EnableNativeSanitizers)" />
<_CoreClrBuildArg Include="-outputrid $(OutputRid)" />
<_CoreClrBuildArg Condition="'$(BuildSubdirectory)' != ''" Include="-subdir $(BuildSubdirectory)" />
</ItemGroup>

<ItemGroup Condition="'$(ClrFullNativeBuild)' != 'true'">
Expand All @@ -52,6 +53,7 @@
<_CoreClrBuildArg Condition="'$(ClrNativeAotSubset)' == 'true'" Include="-component nativeaot" />
<_CoreClrBuildArg Condition="'$(ClrSpmiSubset)' == 'true'" Include="-component spmi" />
<_CoreClrBuildArg Condition="'$(ClrCrossComponentsSubset)' == 'true'" Include="-component crosscomponents" />
<_CoreClrBuildArg Condition="'$(ClrDebugSubset)' == 'true'" Include="-component debug" />
</ItemGroup>

<PropertyGroup>
Expand Down
12 changes: 8 additions & 4 deletions src/libraries/externals.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -85,14 +85,18 @@
<!-- CoreRun is not used for testing anymore, but we still use it for benchmarking and profiling -->
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\corerun*" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\PDB\corerun*" />
<!-- If the build has native sanitizers, copy over the non-sanitized diagnostic binaries so they can be loaded by a debugger -->
</ItemGroup>
<!-- If the build has native sanitizers, copy over the non-sanitized diagnostic binaries so they can be loaded by a debugger -->
<ItemGroup Condition="'$(EnableNativeSanitizers)' != ''">
<RuntimeFiles Remove="$(CoreCLRArtifactsPath)\sharedFramework\*mscordaccore*" />
<RuntimeFiles Remove="$(CoreCLRArtifactsPath)\sharedFramework\*mscordbi*" />
<RuntimeFiles Remove="$(CoreCLRArtifactsPath)\sharedFramework\PDB\*mscordaccore*" />
<RuntimeFiles Remove="$(CoreCLRArtifactsPath)\sharedFramework\PDB\*mscordbi*" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\sharedFramework\*" IsNative="true" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\PDB\*mscordaccore*" IsNative="true" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\PDB\*mscordbi*" IsNative="true" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\unsanitized\sharedFramework\*" IsNative="true" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\unsanitized\PDB\*mscordaccore*" IsNative="true" />
<RuntimeFiles Include="$(CoreCLRArtifactsPath)\$(TargetArchitecture)\unsanitized\PDB\*mscordbi*" IsNative="true" />
</ItemGroup>
<ItemGroup>
<ReferenceCopyLocalPaths Include="@(RuntimeFiles)" />
</ItemGroup>
<ItemGroup Condition="'$(SwapNativeForIL)' == 'true'">
Expand Down
2 changes: 1 addition & 1 deletion src/tests/Common/Directory.Build.targets
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
<SanitizedDiagnosticTools Include="$(LibPrefix)mscordbi$(LibSuffix)" />
<SanitizedDiagnosticTools Include="$(LibPrefix)dbgshim$(LibSuffix)" />
<RunTimeArtifactsExcludeFiles Include="@(SanitizedDiagnosticTools->'%(FileName)%(Extension)')" />
<RunTimeArtifactsIncludeFolders Include="$(TargetArchitecture)/" TargetDir="/" />
<RunTimeArtifactsIncludeFolders Include="$(TargetArchitecture)/unsanitized/" TargetDir="/" />
</ItemGroup>

<ItemGroup Condition="'$(MinimalCoreRoot)' == 'true'">
Expand Down