Skip to content

Add llvm-project to source-build. #14184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
MichaelSimons merged 13 commits into from
Aug 11, 2022
Merged

Add llvm-project to source-build. #14184

MichaelSimons merged 13 commits into from
Aug 11, 2022

Conversation

@crummel
Copy link

@crummel crummel commented Jul 20, 2022 •
edited
Loading

@am11
Copy link
Member

am11 commented Jul 25, 2022

Two things to note about dotnet/llvm-project repository branches:

  • its default branch is dotnet/main (sic). This branch is current sync'd with upstreams v14, and it is used by mono's AOT.
    • mono's llvm-aot uses multiple components of llvm-toolchain; mono runtime, in llvm-aot mode, generates LLVM IR which is then compiled by the actual llvm-toolchain.
  • the other interesting branch is objwriter/12.x, which is based on llvm v12, and used by coreclr's AOT.
    • it is comparatively a tiny component which is built for coreclr's AOT. It does not require cloning and building the entire llvm toolchain. ObjectWriter is basically a brief API exposed by .NET runtime team on top of the existing LLVM's backend assembler implementation; this API helps NaiveAOT mode produce the final objects / binaries for a given platform; the actual native codegen of .NET assemblies takes place in coreclr/jit in the runtime repository. Prior to ObjectWriter, we used to have a handwritten implementation called AsmWriter, which had quite a few shortcomings, thus creation of ObjectWriter API was deemed necessary.

I am not sure if arcade's project dependency model supports "multiple roles" for same <Dependency> based on branch names.

@crummel crummel marked this pull request as ready for review August 10, 2022 20:22
@crummel crummel requested a review from a team as a code owner August 10, 2022 20:22
MichaelSimons
MichaelSimons reviewed Aug 10, 2022
View reviewed changes
MichaelSimons
MichaelSimons approved these changes Aug 11, 2022
View reviewed changes
Copy link
Member

@MichaelSimons MichaelSimons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please open issues/PRs in LLVM to backport all patches and link the issues/PRs in the patches before merging.

@MichaelSimons MichaelSimons enabled auto-merge (squash) August 11, 2022 18:30
@MichaelSimons MichaelSimons merged commit 68a08b0 into dotnet:release/7.0.1xx-preview7 Aug 11, 2022
crummel added a commit to crummel/dotnet_core-sdk that referenced this pull request Aug 12, 2022
@crummel
Add llvm-project to source-build. (dotnet#14184)
12a5d2b 
* Add llvm-project to source-build.

* Add repo references to llvm-project.

* Add source-build metadata.

* Upgrade CI to CentOS Stream 8.

* Fix the image name in the right place.

* Patch llvm to use an MS.Build.Traversal that we have.

* Add backport PR to patch.

* Revert "Upgrade CI to CentOS Stream 8."

This reverts commit 6bc2754.

* Update to avoid new MSBuild warning that causes build failure.

* We use code out of the LLVM directory.  Copy the license file from there to the root to satisfy the license checker.

* Add SourceBuild.props to llvm.

* Update patches with backport PR pointers.
@crummel crummel mentioned this pull request Aug 12, 2022
Merged
MichaelSimons pushed a commit that referenced this pull request Aug 19, 2022
@crummel
Add llvm-project to source-build. (#14184) (#14314)
84d26a0 
* Add llvm-project to source-build.

* Add repo references to llvm-project.

* Add source-build metadata.

* Upgrade CI to CentOS Stream 8.

* Fix the image name in the right place.

* Patch llvm to use an MS.Build.Traversal that we have.

* Add backport PR to patch.

* Revert "Upgrade CI to CentOS Stream 8."

This reverts commit 6bc2754.

* Update to avoid new MSBuild warning that causes build failure.

* We use code out of the LLVM directory.  Copy the license file from there to the root to satisfy the license checker.

* Add SourceBuild.props to llvm.

* Update patches with backport PR pointers.
MichaelSimons added a commit that referenced this pull request Aug 22, 2022
@dotnet-maestro-bot @dotnet-maestro
@MichaelSimons @crummel
[automated] Merge branch 'release/7.0.1xx' => 'main' (#14359)
e0d94df 
* Update dependencies from https://github.com/dotnet/arcade build 20220818.4 (#14354)

[release/7.0.1xx] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.2.0-beta-22410-01 to 1.2.0-beta-22416-01 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.22409.1 to 1.0.0-beta.22415.3 (parent: Microsoft.DotNet.Arcade.Sdk)

* Enable BlazorWasm source-build test (#14346)

* Add llvm-project to source-build. (#14184) (#14314)

* Add llvm-project to source-build.

* Add repo references to llvm-project.

* Add source-build metadata.

* Upgrade CI to CentOS Stream 8.

* Fix the image name in the right place.

* Patch llvm to use an MS.Build.Traversal that we have.

* Add backport PR to patch.

* Revert "Upgrade CI to CentOS Stream 8."

This reverts commit 6bc2754.

* Update to avoid new MSBuild warning that causes build failure.

* We use code out of the LLVM directory.  Copy the license file from there to the root to satisfy the license checker.

* Add SourceBuild.props to llvm.

* Update patches with backport PR pointers.

Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Michael Simons <[email protected]>
Co-authored-by: Chris Rummel <[email protected]>
Co-authored-by: Jason Zhai <[email protected]>
MichaelSimons added a commit that referenced this pull request Aug 26, 2022
@dotnet-maestro-bot @dotnet-maestro
@marcpopMSFT @MichaelSimons @crummel @v-wuzhai @lbussell @MattGal @NikolaMilosavljevic
[automated] Merge branch 'release/7.0.1xx' => 'main' (#14394)
91953ec 
* Update dependencies from https://github.com/dotnet/arcade build 20220812.2 (#14326)

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.CMake.Sdk
 From Version 6.0.0-beta.22411.3 -> To Version 6.0.0-beta.22412.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Update dependencies from https://github.com/dotnet/source-build-reference-packages build 20220819.1

Microsoft.SourceBuild.Intermediate.source-build-reference-packages
 From Version 6.0.0-servicing.22413.1 -> To Version 6.0.0-servicing.22419.1

* Update dependencies from https://github.com/dotnet/arcade build 20220818.4 (#14354)

[release/7.0.1xx] Update dependencies from dotnet/arcade
- Coherency Updates:
  - Microsoft.SourceLink.GitHub: from 1.2.0-beta-22410-01 to 1.2.0-beta-22416-01 (parent: Microsoft.DotNet.Arcade.Sdk)
  - Microsoft.DotNet.XliffTasks: from 1.0.0-beta.22409.1 to 1.0.0-beta.22415.3 (parent: Microsoft.DotNet.Arcade.Sdk)

* Update dependencies from https://github.com/dotnet/source-build-reference-packages build 20220819.2

Microsoft.SourceBuild.Intermediate.source-build-reference-packages
 From Version 6.0.0-servicing.22419.1 -> To Version 6.0.0-servicing.22419.2

* Enable BlazorWasm source-build test (#14346)

* Add llvm-project to source-build. (#14184) (#14314)

* Add llvm-project to source-build.

* Add repo references to llvm-project.

* Add source-build metadata.

* Upgrade CI to CentOS Stream 8.

* Fix the image name in the right place.

* Patch llvm to use an MS.Build.Traversal that we have.

* Add backport PR to patch.

* Revert "Upgrade CI to CentOS Stream 8."

This reverts commit 6bc2754.

* Update to avoid new MSBuild warning that causes build failure.

* We use code out of the LLVM directory.  Copy the license file from there to the root to satisfy the license checker.

* Add SourceBuild.props to llvm.

* Update patches with backport PR pointers.

* Update poison baseline. (#14345)

* Update poison baseline.

* Fix RIDs by replacing with placeholders.

* Exclude a few more non-binary file types from poison checking.

* Ignore empty files when checking for poison.

* Update baseline with new exclusions.

* Poison report can have both portable and non-portable RIDs in it, remove both.

* Update dependencies from https://github.com/dotnet/sdk build 20220823.8 (#14371)

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22416.39 -> To Version 7.0.100-rc.2.22423.8

Dependency coherency updates

Microsoft.WindowsDesktop.App.Ref,VS.Redist.Common.WindowsDesktop.SharedFramework.x64.7.0,VS.Redist.Common.WindowsDesktop.TargetingPack.x64.7.0,Microsoft.NETCore.App.Runtime.win-x64,runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.NETCore.App.Ref,VS.Redist.Common.NetCore.TargetingPack.x64.7.0,Microsoft.NETCore.App.Host.win-x64,Microsoft.NETCore.DotNetHostResolver,Microsoft.NETCore.Platforms,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.WindowsDesktop.App.Runtime.win-x64,Microsoft.Dotnet.WinForms.ProjectTemplates,Microsoft.WindowsDesktop.App.Runtime.win-x64,Microsoft.DotNet.Wpf.ProjectTemplates,Microsoft.FSharp.Compiler,Microsoft.SourceBuild.Intermediate.fsharp,Microsoft.Net.Compilers.Toolset,Microsoft.Build,NuGet.Build.Tasks,VS.Redist.Common.NetCore.SharedFramework.x64.7.0,Microsoft.NET.Workload.Emscripten.Manifest-7.0.100
 From Version 7.0.0-rc.1.22379.3 -> To Version 7.0.0-rc.2.22422.9 (parent: Microsoft.NET.Sdk

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Update dependencies from https://github.com/dotnet/sdk build 20220823.25

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.13 -> To Version 7.0.100-rc.2.22423.25

Dependency coherency updates

Microsoft.WindowsDesktop.App.Ref,VS.Redist.Common.WindowsDesktop.SharedFramework.x64.7.0,VS.Redist.Common.WindowsDesktop.TargetingPack.x64.7.0,Microsoft.WindowsDesktop.App.Runtime.win-x64,Microsoft.Dotnet.WinForms.ProjectTemplates,Microsoft.WindowsDesktop.App.Runtime.win-x64,Microsoft.DotNet.Wpf.ProjectTemplates
 From Version 7.0.0-rc.2.22422.9 -> To Version 7.0.0-rc.2.22423.4 (parent: Microsoft.NET.Sdk

* Dependency coherency updates (#14365)

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter
 From Version 1.0.0-alpha.1.22364.1 -> To Version 1.0.0-alpha.1.22364.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Update source-build build images (#14321)

* Fix aspnetcore offline build when nodejs is installed

* Remove centos7 and 8, replace with stream 8

* Change Fedora 33 container to Fedora 36

* Revert "Change Fedora 33 container to Fedora 36"

* Remove unused debian container definition

* Add extra packages needed for bootstrapping arm64

* Use PackageDownloads for bootstrapping PSB artifacts and update prebuilts version

Co-authored-by: Chris Rummel <[email protected]>

* Prevent System.Memory package downgrade in razor-compiler (#14384) (#14391)

* Revert "Add razor-compiler patch to fix bootstrap build (#14379)"

This reverts commit 1ebb3eb.

* Add razor-compiler patch to prevent System.Memory downgrades

* Add roslyn patch to lift System.Memory version

* Use -svc pool provider in release branches

* [release/7.0.1xx] Update dependencies from dotnet/sdk (#14381)

* Update dependencies from https://github.com/dotnet/sdk build 20220823.30

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.25 -> To Version 7.0.100-rc.2.22423.30

Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.NET.ILLink.Tasks
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Update dependencies from https://github.com/dotnet/sdk build 20220823.32

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.25 -> To Version 7.0.100-rc.2.22423.32

Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.NET.ILLink.Tasks
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Update dependencies from https://github.com/dotnet/sdk build 20220824.4

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.25 -> To Version 7.0.100-rc.2.22424.4

Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.NET.ILLink.Tasks,Microsoft.Net.Compilers.Toolset
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Update dependencies from https://github.com/dotnet/sdk build 20220824.9

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.25 -> To Version 7.0.100-rc.2.22424.9

Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.NET.ILLink.Tasks,Microsoft.Net.Compilers.Toolset
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Update dependencies from https://github.com/dotnet/sdk build 20220824.14

Microsoft.DotNet.Common.ItemTemplates , Microsoft.DotNet.MSBuildSdkResolver , Microsoft.NET.Sdk , Microsoft.TemplateEngine.Cli
 From Version 7.0.100-rc.2.22423.25 -> To Version 7.0.100-rc.2.22424.14

Dependency coherency updates

runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter,Microsoft.AspNetCore.App.Ref,Microsoft.AspNetCore.App.Ref.Internal,Microsoft.AspNetCore.App.Runtime.win-x64,VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0,dotnet-dev-certs,dotnet-user-jwts,dotnet-user-secrets,Microsoft.NET.ILLink.Tasks,Microsoft.Net.Compilers.Toolset
 From Version 1.0.0-alpha.1.22411.1 -> To Version 1.0.0-alpha.1.22411.1 (parent: Microsoft.NETCore.App.Runtime.win-x64

* Resolve llvm source-build patch conflict

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: MichaelSimons <[email protected]>

* Remove XDT dependency and patch (#14395)

* [release/7.0.1xx] Update dependencies from dotnet/sdk (#14396)

[release/7.0.1xx] Update dependencies from dotnet/sdk
- Coherency Updates:
  - Microsoft.WindowsDesktop.App.Ref: from 7.0.0-rc.2.22423.4 to 7.0.0-rc.2.22424.8 (parent: Microsoft.NET.Sdk)
  - VS.Redist.Common.WindowsDesktop.SharedFramework.x64.7.0: from 7.0.0-rc.2.22423.4 to 7.0.0-rc.2.22424.8 (parent: Microsoft.NET.Sdk)
  - VS.Redist.Common.WindowsDesktop.TargetingPack.x64.7.0: from 7.0.0-rc.2.22423.4 to 7.0.0-rc.2.22424.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Ref: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Ref.Internal: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - Microsoft.AspNetCore.App.Runtime.win-x64: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - VS.Redist.Common.AspNetCore.SharedFramework.x64.7.0: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - dotnet-dev-certs: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - dotnet-user-jwts: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - dotnet-user-secrets: from 7.0.0-rc.2.22423.16 to 7.0.0-rc.2.22424.4 (parent: Microsoft.NET.Sdk)
  - Microsoft.WindowsDesktop.App.Runtime.win-x64: from 7.0.0-rc.2.22423.4 to 7.0.0-rc.2.22424.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.Dotnet.WinForms.ProjectTemplates: from 7.0.0-rc.2.22423.3 to 7.0.0-rc.2.22424.12 (parent: Microsoft.WindowsDesktop.App.Runtime.win-x64)
  - Microsoft.WindowsDesktop.App.Runtime.win-x64: from 7.0.0-rc.2.22423.4 to 7.0.0-rc.2.22424.8 (parent: Microsoft.NET.Sdk)
  - Microsoft.DotNet.Wpf.ProjectTemplates: from 7.0.0-rc.2.22423.2 to 7.0.0-rc.2.22424.8 (parent: Microsoft.WindowsDesktop.App.Runtime.win-x64)
  - Microsoft.Net.Compilers.Toolset: from 4.4.0-2.22423.18 to 4.4.0-2.22424.9 (parent: Microsoft.NET.Sdk)

* Update dependencies from https://github.com/dotnet/source-build-externals build 20220824.1 (#14401)

[release/7.0.1xx] Update dependencies from dotnet/source-build-externals

* Update Versions.Details.xml source-build metadata (#14375)

* Update dependencies from https://github.com/dotnet/test-templates build 20220825.1 (#14400)

Microsoft.DotNet.Test.ProjectTemplates.5.0 , Microsoft.DotNet.Test.ProjectTemplates.6.0 , Microsoft.DotNet.Test.ProjectTemplates.7.0
 From Version 1.0.2-beta4.22409.1 -> To Version 1.0.2-beta4.22425.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Update source-build-externals (#14405)

Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Marc Paine <[email protected]>
Co-authored-by: Michael Simons <[email protected]>
Co-authored-by: Chris Rummel <[email protected]>
Co-authored-by: v-wuzhai <[email protected]>
Co-authored-by: Jason Zhai <[email protected]>
Co-authored-by: Logan Bussell <[email protected]>
Co-authored-by: Matt Galbraith <[email protected]>
Co-authored-by: Nikola Milosavljevic <[email protected]>
Co-authored-by: Matt Galbraith <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MichaelSimons MichaelSimons MichaelSimons approved these changes

@lbussell lbussell lbussell approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@crummel @am11 @MichaelSimons @lbussell