dotnet-watch tests failing

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=629827
Build error leg or test failing: dotnet-watch.Tests.dll.1.WorkItemExecution
Pull request: #39888

Error Message

##[error].packages/microsoft.dotnet.helix.sdk/8.0.0-beta.24177.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(89,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item dotnet-watch.Tests.dll.1 in job 82728a56-9430-4356-ab82-1c21670792a7 has failed

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item dotnet-watch\\.Tests\\.dll\\.1 in job [a-z0-9\\-]+ has failed",
  "BuildRetry": true,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=629827
Error message validated: [Work item dotnet-watch\.Tests\.dll\.1 in job [a-z0-9\-]+ has failed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/4/2024 9:51:10 AM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1230968	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52135
1230151	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52135
1229849	dotnet/sdk	Run FullFramework Tests	Log	#52148
2859464	dotnet-sdk	Run FullFramework Tests	Log	#56049
2859463	dotnet-sdk	Run TestBuild Tests	Log	#56048
1229178	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52053
1227774	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52053
1227484	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51307
1225970	dotnet/sdk	Run FullFramework Tests	Log	#52099
1225956	dotnet/sdk	Run FullFramework Tests	Log	#52098
1225955	dotnet/sdk	Run FullFramework Tests	Log	#52097
1227598	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52053
1226921	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51088
1226175	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52028
2857861	dotnet-sdk	Run TestBuild Tests	Log	#55976
2857862	dotnet-sdk	Run FullFramework Tests	Log	#55975
1224967	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52028
1224826	dotnet/sdk	Run TestBuild Tests	Log	#52007
2856664	dotnet-sdk	Run TestBuild Tests	Log	#55853
1223955	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51307
1223285	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51307
1222286	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51727
1222071	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51068
1222239	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51068
1221516	dotnet/sdk	Run TestBuild Tests	Log	#52007
1221157	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52033
1221118	dotnet/sdk	Run TestBuild Tests	Log	#52013
2854049	dotnet-sdk	Run TestBuild Tests	Log	#55844
1220569	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51068
1220056	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51881
1219988	dotnet/sdk	Run TestBuild Tests	Log	#52008
1219768	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51881
1219830	dotnet/sdk	Run FullFramework Tests	Log	#52006
1219822	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52005
1219420	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51632
1218206	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1219523	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51419
1218232	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51935
1218297	dotnet/sdk	Run TestBuild Tests	Log	#51985
1216834	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1216781	dotnet/sdk	Run TestBuild Tests	Log	#51910
1216911	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51935
1216549	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51624
1216353	dotnet/sdk	Run TestBuild Tests	Log	#51891
1215855	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51935
1213145	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1215739	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51068
1215251	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51947
1214650	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51942
1214670	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51935
1213624	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51881
1213293	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51881
1213207	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51918
1212185	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51881
1212157	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1212146	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51857
1212001	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1210799	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1210165	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1210104	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51861
1209844	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51826
1209743	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51826
1209507	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1209276	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51847
1209240	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51844
1209262	dotnet/sdk	Run TestBuild Tests	Log	#51846
1209239	dotnet/sdk	Run TestBuild Tests	Log	#51843
1208208	dotnet/sdk	Run TestBuild Tests	Log	#51816
1208052	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51727
1207838	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51727
1206878	dotnet/sdk	🟣 Run FullFramework Tests	Log	#50864
1206801	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51419
1206632	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51794
1206490	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51790
1205092	dotnet/sdk	Run TestBuild Tests	Log	#51698
1205101	dotnet/sdk	Run TestBuild Tests	Log	#51705
1203943	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51727
1199032	dotnet/sdk	Run TestBuild Tests	Log	#51535
1203612	dotnet/sdk	🟣 Run FullFramework Tests	Log	#51727

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	21	79

[marcpopMSFT]

@tmat we still see this fairly frequently. Any luck? Here's a recent one from a codeflow PR:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-41616-merge-58511284b77c40778a/dotnet-watch.Tests.dll.1/1/console.418de7ca.log?helixlogtype=result

[tmat]

No luck yet. This might help: https://github.com/dotnet/sdk/blob/d45e40d665e647804de03d2b4eec2aae9c488cb5/test/dotnet-watch.Tests/xunit.runner.json

It will print out test info every 20 seconds a test is running.

[marcpopMSFT]

@tmat another failure this morning shows a bunch of build processes for watch apps still running... https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-sdk-refs-pull-41958-merge-84357342ee154a5088/dotnet-watch.Tests.dll.2/1/console.3dddc463.log?helixlogtype=result

[tmat]

Looks like that's in 8.0 branch. Porting test diagnostics from 9: #41963

[Forgind]

This has been making one of my PRs fail today, so I looked into it a bit. Based on when this issue was created and which tests are failing, I suspect this PR:
#39618

I asked jasonmalinowski whether the MSBuildWorkspace is supposed to support F# because I'd found a code snippet that suggested that it shouldn't, and he said it doesn't and never did. Did that PR make us use MSBuildWorkspace in a different way than we had before?

[Forgind]

The changes to HotReloadDotNetWatcher look most suspicious to me. It looks like a substantial change to how we call MSBuild.