Use VMR instead of the tarball for Source-Build builds

[premun]

Changes to CI

For PRs, the Source-Build leg that was running inside of the Build stage is now moved to a separate stage but runs more or less the same: https://dev.azure.com/dnceng-public/public/_build/results?buildId=97509&view=results
Instead of creating the tarball, we are building the dotnet/dotnet repo there.

For internal rolling builds, we are taking this pipeline and merging it into dotnet-installer-official-ci.
So it's one extra stage that runs pretty quick (faster than the Build stage by far).
It won't be creating and pushing the tarball artifact anymore though.

Once the rolling build is finished, there won't be no more source-build-build pipeline but instead dotnet-dotnet-official-ci which will build the dotnet/dotnet repo again instead of the tarball that was originally produced from the rolling build.
The MSFT SDK from the installer build will still be consumed by it though.

More details dotnet/arcade#10677

Example builds

• PR build - https://dev.azure.com/dnceng-public/public/_build/results?buildId=95119
• dotnet-dotnet-source-build (internal rolling) - https://dev.azure.com/dnceng/internal/_build/results?buildId=2054513
• dotnet-installer-official-ci (manual run) - https://dev.azure.com/dnceng/internal/_build/results?buildId=2054548
  (runs the whole matrix as opposed to the PR build)
• Internal dotnet-installer-official-ci rolling build will only run the VMR synchronization stage outside of what it does now
• Public dotnet-installer build will not run any extra steps as compared to today's

To discuss in the PR

• We could theoretically go back to the way the matrix was. After a few days of refactoring, I ended up with more or less a very similar setup as there was initially.
• File names - everything is vmr-source-build.yml (pipeline/stage/job/steps). I am open to something different, just don't know what that is.

[premun]

@MichaelSimons TLDR of my today's attempts + status quo:

• I wasn't able to utilize the matrix but I managed to simplify the list of the template calls so it's a bit more concise.
• We need to decide whether we want to keep the Linux build stage as it can mean we trigger the official VMR pipeline 30 minutes earlier.
• There will be no separate VMR sync validation run during the PR as it is part of the VMR leg. This is a must though so in case we would be temporarily disabling the VMR build in PRs, we would ideally add the vmr-synchronization.yml back as it catches many problems with the VMR tooling / VMR patches / etc.. This is a silent contract now though and I am hesitant whether we shouldn't run one more VMR sync validation on the side either way?

[MichaelSimons]

I wasn't able to utilize the matrix but I managed to simplify the list of the template calls so it's a bit more concise.

It looks better. Given the AzDO limitations I think your approach is good and we should go with it.

We need to decide whether we want to keep the Linux build stage as it can mean we trigger the official VMR pipeline 30 minutes earlier.

I think it should be removed.

There will be no separate VMR sync validation run during the PR as it is part of the VMR leg. This is a must though so in case we would be temporarily disabling the VMR build in PRs, we would ideally add the vmr-synchronization.yml back as it catches many problems with the VMR tooling / VMR patches / etc.. This is a silent contract now though and I am hesitant whether we shouldn't run one more VMR sync validation on the side either way?

Per our conversation, I agree is should be commented out with a note that the vmr-synchronization stage is necessary if the source-build leg is disabled.

[premun]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).