Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
18ef824
[ci] Migrate to the 1ES template
pjcollins Feb 21, 2024
2de0a12
Try to fix variable import
pjcollins Feb 21, 2024
b9eff07
Use rooted template paths
pjcollins Feb 21, 2024
3bf4782
Rooted template paths take 2
pjcollins Feb 21, 2024
a5818bd
Use 1es publish for linux build
pjcollins Feb 21, 2024
743635c
Pass use1ESTemplate=true to all signing jobs
pjcollins Feb 21, 2024
0c3d9f2
Use 'output: nuget' instead of NuGetCommand@2
pjcollins Feb 21, 2024
c2dce75
Update sdl repos to scan
pjcollins Feb 21, 2024
ebe16bf
Update pools
pjcollins Feb 21, 2024
dabedab
Fixing pools, guardian suppressions
pjcollins Feb 21, 2024
50fc87e
Update monodroid guardian files
pjcollins Feb 21, 2024
fcb0c2f
More windows parallelization, use pipeline outputs for macos build
pjcollins Feb 22, 2024
398dae9
Remove setup-linux, update nightly
pjcollins Feb 22, 2024
35c5864
bump monodroid, fix mac build source path
pjcollins Feb 22, 2024
e0cc0ab
Update result publishing and timeout
pjcollins Feb 22, 2024
b68f2e6
Merge remote-tracking branch 'origin/main' into dev/pjc/1estemp
pjcollins Feb 22, 2024
d174ca2
Restore multi-language policheck scan
pjcollins Feb 22, 2024
c6ada8e
Rename gdnsuppress for policheck template
pjcollins Feb 22, 2024
23808ee
Update source.gdnsuppress
pjcollins Feb 22, 2024
acd8ef6
Update guardian files
pjcollins Feb 23, 2024
9385185
Remove dupe folder
pjcollins Feb 23, 2024
6ee0182
Merge remote-tracking branch 'origin/main' into dev/pjc/1estemp
pjcollins Feb 26, 2024
c997c20
Update sdl task params
pjcollins Feb 26, 2024
0b90087
Use yaml templates main
pjcollins Feb 27, 2024
1f323ee
Test new windows pool
pjcollins Feb 28, 2024
20352a6
Merge remote-tracking branch 'origin/main' into dev/pjc/1estemp
pjcollins Feb 28, 2024
80572d0
Use new win pool
pjcollins Feb 28, 2024
45f3b6f
Invoke boots with pwsh, diagnostic install
pjcollins Feb 28, 2024
fbff670
Install dotnet tools locally and prepend path
pjcollins Feb 28, 2024
fac7da6
Try full tool path
pjcollins Feb 28, 2024
fbe295f
Use full tool paths
pjcollins Feb 28, 2024
b65dea8
Try to reenable static macos PR build pool
pjcollins Feb 29, 2024
536df9a
Try MAUI-DevDiv pool
pjcollins Feb 29, 2024
03532f3
Test new MAUI-1ESPT pool
pjcollins Feb 29, 2024
5bf687c
Update names, dotnet tool install path and lookup
pjcollins Feb 29, 2024
ff8af4f
Fix macos pool condition
pjcollins Feb 29, 2024
a839e16
Testing powershell on image
pjcollins Mar 1, 2024
a82985e
Try to add powershell to path for maui test job
pjcollins Mar 1, 2024
18d700a
Simplify macos PR build pool condition
pjcollins Mar 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
[ci] Migrate to the 1ES template
Context: https://aka.ms/1espt

The build pipeline has been updated to extend the 1ES pipeline template,
which will keep the pipeline up to date with the latest compliance and
security requirements. Compliance tasks and scans will run automatically
as part of artifact upload steps.
  • Loading branch information
pjcollins committed Feb 21, 2024
commit 18ef8241414a0f4aed429cf19ada929a2c475ea6
12 changes: 0 additions & 12 deletions build-tools/automation/azure-pipelines-nightly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,6 @@ stages:
steps:
- template: yaml-templates/setup-test-environment.yaml
parameters:
installLegacyDotNet: false
restoreNUnitConsole: false
updateMono: false
xaprepareScenario: EmulatorTestDependencies
jdkTestFolder: $(JAVA_HOME_11_X64)

Expand Down Expand Up @@ -200,9 +197,6 @@ stages:
- template: yaml-templates/setup-test-environment.yaml
parameters:
installTestSlicer: true
installLegacyDotNet: false
restoreNUnitConsole: false
updateMono: false
xaprepareScenario: EmulatorTestDependencies
jdkTestFolder: $HOME/android-toolchain/jdk-17

Expand Down Expand Up @@ -258,9 +252,6 @@ stages:
- template: yaml-templates/setup-test-environment.yaml
parameters:
installTestSlicer: true
installLegacyDotNet: false
restoreNUnitConsole: false
updateMono: false
xaprepareScenario: EmulatorTestDependencies
jdkTestFolder: $HOME/android-toolchain/jdk-17

Expand Down Expand Up @@ -303,9 +294,6 @@ stages:
- template: yaml-templates/setup-test-environment.yaml
parameters:
installApkDiff: false
installLegacyDotNet: false
restoreNUnitConsole: false
updateMono: false

- task: DownloadPipelineArtifact@2
displayName: Download binutils pdbs
Expand Down
1,171 changes: 541 additions & 630 deletions build-tools/automation/azure-pipelines.yaml

Large diffs are not rendered by default.

35 changes: 9 additions & 26 deletions build-tools/automation/yaml-templates/apk-instrumentation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,41 +9,24 @@ parameters:
testResultsFormat: NUnit
artifactSource: ""
artifactFolder: ""
useDotNet: true
condition: succeeded()
retryCountOnTaskFailure: 1

steps:
- ${{ if eq(parameters.useDotNet, false) }}:
- task: MSBuild@1
- template: run-dotnet-preview.yaml@self
parameters:
configuration: ${{ parameters.buildConfiguration }}
xaSourcePath: ${{ parameters.xaSourcePath }}
displayName: run ${{ parameters.testName }}
inputs:
solution: ${{ parameters.project }}
configuration: ${{ parameters.configuration }}
msbuildArguments: >-
/restore
/t:RunTestApp
/bl:${{ parameters.xaSourcePath }}/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog
${{ parameters.extraBuildArgs }}
project: ${{ parameters.project }}
arguments: >-
-t:RunTestApp
-bl:${{ parameters.xaSourcePath }}/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog
-v:n -c ${{ parameters.configuration }} ${{ parameters.extraBuildArgs }}
condition: ${{ parameters.condition }}
continueOnError: true
retryCountOnTaskFailure: ${{ parameters.retryCountOnTaskFailure }}

- ${{ if eq(parameters.useDotNet, true) }}:
- template: run-dotnet-preview.yaml
parameters:
configuration: ${{ parameters.buildConfiguration }}
xaSourcePath: ${{ parameters.xaSourcePath }}
displayName: run ${{ parameters.testName }}
project: ${{ parameters.project }}
arguments: >-
-t:RunTestApp
-bl:${{ parameters.xaSourcePath }}/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog
-v:n -c ${{ parameters.configuration }} ${{ parameters.extraBuildArgs }}
condition: ${{ parameters.condition }}
continueOnError: true
retryCountOnTaskFailure: ${{ parameters.retryCountOnTaskFailure }}

- script: >
DEST="$(Build.StagingDirectory)/Test${{ parameters.configuration }}/${{ parameters.artifactFolder }}/" &&
mkdir -p "$DEST" &&
Expand Down
36 changes: 6 additions & 30 deletions build-tools/automation/yaml-templates/build-linux.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
parameters:
buildPoolName: $(LinuxBuildPoolName)
buildPoolImage: $(LinuxBuildPoolImage)
buildResultArtifactName: Build Results - Linux
checkoutCommit: ''
checkoutPath: 's/xamarin-android'
Expand All @@ -24,8 +22,9 @@ stages:
- job: ${{ parameters.jobName }}
displayName: ${{ parameters.jobDisplayName }}
pool:
name: ${{ parameters.buildPoolName }}
vmImage: ${{ parameters.buildPoolImage }}
name: AzurePipelines-EO
vmImage: $(LinuxPoolImage1ESPT)
os: linux
timeoutInMinutes: 180
workspace:
clean: all
Expand All @@ -44,7 +43,7 @@ stages:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#checkout-path
- checkout: maui

- template: setup-ubuntu.yaml
- template: setup-ubuntu.yaml@self

- ${{ if ne(variables['System.PullRequest.IsFork'], 'True') }}:
- checkout: monodroid
Expand Down Expand Up @@ -90,33 +89,10 @@ stages:
artifactName: ${{ parameters.nugetArtifactName }}
targetPath: $(System.DefaultWorkingDirectory)/xamarin-android/bin/Build$(XA.Build.Configuration)/nuget-linux

- powershell: |
[IO.Directory]::CreateDirectory("$(Build.StagingDirectory)/empty")
[IO.Directory]::CreateDirectory("$(Build.StagingDirectory)/sbom-components")
displayName: create SBOM directories
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))

- task: AzureArtifacts.manifest-generator-task.manifest-generator-task.ManifestGeneratorTask@0
displayName: generate components SBOM
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))
inputs:
BuildDropPath: $(Build.StagingDirectory)/empty
BuildComponentPath: $(System.DefaultWorkingDirectory)/xamarin-android
ManifestDirPath: $(Build.StagingDirectory)/sbom-components
PackageName: .NET Android
Verbosity: Verbose

- task: PublishBuildArtifacts@1
displayName: publish components SBOM
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))
inputs:
artifactName: sbom-components-linux
pathToPublish: $(Build.StagingDirectory)/sbom-components

- template: upload-results.yaml
- template: upload-results.yaml@self
parameters:
xaSourcePath: $(System.DefaultWorkingDirectory)/xamarin-android
artifactName: ${{ parameters.buildResultArtifactName }}
includeBuildResults: true

- template: fail-on-issue.yaml
- template: fail-on-issue.yaml@self
36 changes: 6 additions & 30 deletions build-tools/automation/yaml-templates/build-macos.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
parameters:
buildPoolName: $(MacBuildPoolName)
buildPoolImage: $(MacBuildPoolImage)
buildResultArtifactName: Build Results - macOS
checkoutCommit: ''
checkoutPath: 's/xamarin-android'
Expand Down Expand Up @@ -28,9 +26,10 @@ stages:
- job: ${{ parameters.jobName }}
displayName: ${{ parameters.jobDisplayName }}
pool:
name: ${{ parameters.buildPoolName }}
vmImage: ${{ parameters.buildPoolImage }}
${{ if or(and(ne(variables['Build.DefinitionName'],'Xamarin.Android'), ne(variables['Build.DefinitionName'], 'Xamarin.Android-Private'), ne(variables['Build.DefinitionName'], 'xamarin.megapipeline')), eq(variables['Build.Reason'], 'PullRequest')) }}:
name: $(MacBuildPoolName)
vmImage: $(MacBuildPoolImage)
os: macOS
${{ if ne(variables['MicroBuildSignType'], 'Real') }}:
demands: macOS.Name -equals Monterey
timeoutInMinutes: 240
cancelTimeoutInMinutes: 5
Expand All @@ -44,37 +43,14 @@ stages:
path: ${{ parameters.checkoutPath }}
persistCredentials: ${{ parameters.checkoutPersistCredentials }}

- template: commercial-build.yaml
- template: commercial-build.yaml@self
parameters:
installerArtifactName: ${{ parameters.installerArtifactName }}
nugetArtifactName: ${{ parameters.nugetArtifactName }}
testAssembliesArtifactName: ${{ parameters.testAssembliesArtifactName }}
windowsToolchainPdbArtifactName: ${{ parameters.windowsToolchainPdbArtifactName }}

- powershell: |
[IO.Directory]::CreateDirectory("$(Build.StagingDirectory)/empty")
[IO.Directory]::CreateDirectory("$(Build.StagingDirectory)/sbom-components")
displayName: create SBOM directories
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))

- task: AzureArtifacts.manifest-generator-task.manifest-generator-task.ManifestGeneratorTask@0
displayName: generate components SBOM
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))
inputs:
BuildDropPath: $(Build.StagingDirectory)/empty
BuildComponentPath: $(System.DefaultWorkingDirectory)/xamarin-android
ManifestDirPath: $(Build.StagingDirectory)/sbom-components
PackageName: .NET Android
Verbosity: Verbose

- task: PublishBuildArtifacts@1
displayName: publish components SBOM
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))
inputs:
artifactName: sbom-components-macos
pathToPublish: $(Build.StagingDirectory)/sbom-components

- template: upload-results.yaml
- template: upload-results.yaml@self
parameters:
xaSourcePath: $(System.DefaultWorkingDirectory)/xamarin-android
artifactName: ${{ parameters.buildResultArtifactName }}
Expand Down
22 changes: 12 additions & 10 deletions build-tools/automation/yaml-templates/build-windows.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
parameters:
buildPool: $(1ESWindowsPool)
buildResultArtifactName: Build Results - Windows
checkoutCommit: ''
checkoutPath: ''
Expand All @@ -23,7 +22,10 @@ stages:
# Check - "Xamarin.Android (Windows > Build & Smoke Test)"
- job: ${{ parameters.jobName }}
displayName: ${{ parameters.jobDisplayName }}
pool: ${{ parameters.buildPool }}
pool:
name: AzurePipelines-EO
image: $(WindowsPoolImage1ESPT)
os: windows
timeoutInMinutes: 360
steps:
- template: sdk-unified/steps/checkout/v1.yml@yaml-templates
Expand All @@ -33,15 +35,15 @@ stages:
path: ${{ parameters.checkoutPath }}
persistCredentials: ${{ parameters.checkoutPersistCredentials }}

- template: kill-processes.yaml
- template: kill-processes.yaml@self

- template: clean.yaml
- template: clean.yaml@self

- script: |
echo ##vso[task.setvariable variable=JI_JAVA_HOME]%JAVA_HOME_17_X64%
displayName: set JI_JAVA_HOME to $(JAVA_HOME_17_X64)

- template: use-dot-net.yaml
- template: use-dot-net.yaml@self
parameters:
remove_dotnet: true

Expand All @@ -52,7 +54,7 @@ stages:
arguments: '-c $(XA.Build.Configuration) -t:Prepare --no-restore -p:AutoProvision=true -bl:$(System.DefaultWorkingDirectory)\bin\Build$(XA.Build.Configuration)\dotnet-build-prepare.binlog'

# Build, pack .nupkgs, and extract workload packs to dotnet preview test directory
- template: run-dotnet-preview.yaml
- template: run-dotnet-preview.yaml@self
parameters:
project: Xamarin.Android.sln
arguments: >-
Expand All @@ -61,12 +63,12 @@ stages:
displayName: Build Solution
continueOnError: false

- template: install-global-tool.yaml
- template: install-global-tool.yaml@self
parameters:
toolName: apkdiff
version: $(ApkDiffToolVersion)

- template: run-nunit-tests.yaml
- template: run-nunit-tests.yaml@self
parameters:
testRunTitle: Smoke MSBuild Tests - Windows Dotnet Build
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\$(DotNetStableTargetFramework)\Xamarin.Android.Build.Tests.dll
Expand All @@ -79,9 +81,9 @@ stages:
filename: dotnet-local.cmd
arguments: build samples\HelloWorld\HelloWorld\HelloWorld.DotNet.csproj

- template: upload-results.yaml
- template: upload-results.yaml@self
parameters:
artifactName: ${{ parameters.buildResultArtifactName }}
includeBuildResults: true

- template: fail-on-issue.yaml
- template: fail-on-issue.yaml@self
14 changes: 7 additions & 7 deletions build-tools/automation/yaml-templates/commercial-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ steps:
- script: echo "##vso[task.setvariable variable=JI_JAVA_HOME]$HOME/android-toolchain/jdk-17"
displayName: set JI_JAVA_HOME

- template: use-dot-net.yaml
- template: use-dot-net.yaml@self
parameters:
remove_dotnet: true

Expand Down Expand Up @@ -72,7 +72,7 @@ steps:
displayName: CodeQL 3000 Finalize
condition: and(succeededOrFailed(), eq(variables['Codeql.Enabled'], 'true'), eq(variables['Build.SourceBranch'], 'refs/heads/main'))

- template: install-microbuild-tooling.yaml
- template: install-microbuild-tooling.yaml@self
parameters:
condition: and(succeeded(), eq(variables['MicroBuildSignType'], 'Real'))

Expand Down Expand Up @@ -121,21 +121,21 @@ steps:
/p:MicroBuildOverridePluginDirectory=$(Build.StagingDirectory)/MicroBuild/Plugins
/bl:${{ parameters.xaSourcePath }}/bin/Build$(XA.Build.Configuration)/sign-bu-ex.binlog

- template: remove-microbuild-tooling.yaml
- template: remove-microbuild-tooling.yaml@self
parameters:
condition: and(succeededOrFailed(), eq(variables['MicroBuildSignType'], 'Real'))

- script: make create-installers CONFIGURATION=$(XA.Build.Configuration) MSBUILD_ARGS='${{ parameters.makeMSBuildArgs }}'
workingDirectory: ${{ parameters.xaSourcePath }}
displayName: make create-installers

- task: PublishPipelineArtifact@1
- task: 1ES.PublishPipelineArtifact@1
displayName: upload nupkgs
inputs:
artifactName: ${{ parameters.nugetArtifactName }}
targetPath: ${{ parameters.xaSourcePath }}/bin/Build$(XA.Build.Configuration)/nuget-unsigned

- task: PublishPipelineArtifact@1
- task: 1ES.PublishPipelineArtifact@1
displayName: upload test assemblies
inputs:
artifactName: ${{ parameters.testAssembliesArtifactName }}
Expand All @@ -148,13 +148,13 @@ steps:
workingDirectory: ${{ parameters.xaSourcePath }}
displayName: zip Windows toolchain pdb files

- task: PublishPipelineArtifact@1
- task: 1ES.PublishPipelineArtifact@1
displayName: upload Windows toolchain pdb files
inputs:
artifactName: ${{ parameters.windowsToolchainPdbArtifactName }}
targetPath: ${{ parameters.xaSourcePath }}/bin/Build$(XA.Build.Configuration)/windows-toolchain-pdb

- task: PublishPipelineArtifact@1
- task: 1ES.PublishPipelineArtifact@1
displayName: upload build tools inventory
inputs:
artifactName: AndroidBuildToolsInventory
Expand Down
Loading