[release/7.0][wasm-mt] Create a Microsoft.NET.Runtime.Emscripten.Cache

[lambdageek]

Spin out the emscripten system libraries cache into a separate nupkg.

• Split up the Emscripten SDK by moving the system libraries cache into a separate nupkg.

• make a new Emscripten Cache SDK that includes just the cache directory

• the Cache SDK overrides the WasmCachePath (if it's not set by the user) to point into the Cache SDK

• we leave the mt multithreaded libraries in the cache. (But we remove the ww WebWorker (non-pthread) Emscripten API libraries)

Addresses dotnet/runtime#75263

[lambdageek]

Using -L doesn't work because emscripten is very particular about how it looks for libraries that it considers "system" libs. There's a python script that isn't very flexible.

But, we can change the directory where the cache resides.

And locally, if I exclude the *-ww-*.a libraries (since we're not using the non-pthreads Emscripten WebWorkers APIs), the cache dir is only 173 megs.

So instead of making a "Emsciprten.Threading.Sdk" package. we could make an "Emscripten.Cache.Sdk" package instead and put all the libs (threaded and non-threaded) in there, and use the nice WasmCachePath SDK property to override the EM_CACHE environment variable that we set when doing the linking.

This appears to work locally:

  <PropertyGroup>
    <EmscriptenThreadingSdkCacheDir>/home/aleksey/working/dotnet-runtime/src/mono/wasm/emsdk/upstream/emscripten/cache/</EmscriptenThreadingSdkCacheDir>
  </PropertyGroup>

  <Target Name="_EmscriptenThreadingPrependLinkFlags" BeforeTargets="_PrepareForWasmBuildNative" Condition="'$(WasmEnableThreads)' == 'true'">
    <PropertyGroup>
      <WasmCachePath>$(EmscriptenThreadingSdkCacheDir)</WasmCachePath>
    </PropertyGroup>
  </Target>

[lewing]

@joeloff if emsdk adds a new pack is that going to cause you issues?

[lambdageek]

$ ls -lh artifacts/packages/Release/Shipping/
total 271M
-rw-r--r-- 1 aleksey aleksey  50M Sep  9 13:05 Microsoft.NET.Runtime.Emscripten.3.1.12.Cache.Sdk.linux-x64.7.0.0-dev.nupkg
-rw-r--r-- 1 aleksey aleksey  25M Sep  9 13:04 Microsoft.NET.Runtime.Emscripten.3.1.12.Node.linux-x64.7.0.0-dev.nupkg
-rw-r--r-- 1 aleksey aleksey 196M Sep  9 13:04 Microsoft.NET.Runtime.Emscripten.3.1.12.Sdk.linux-x64.7.0.0-dev.nupkg
-rw-r--r-- 1 aleksey aleksey 9.5K Sep  9 13:05 Microsoft.NET.Workload.Emscripten.net6.Manifest-7.0.100.7.0.0-dev.nupkg
-rw-r--r-- 1 aleksey aleksey 9.7K Sep  9 13:05 Microsoft.NET.Workload.Emscripten.net7.Manifest-7.0.100.7.0.0-dev.nupkg

So the emscripten SDK is now 196 megs, and the cache sdk is 50 megs

And the Cache SDK contents are listed here https://gist.github.com/lambdageek/53991645623340551e65c4be4213bfa0

[lambdageek]

@lewing pushed the workload manifest changes, too.

[lewing]

maybe Microsoft.NET.Runtime.Emscripten.Cache.Sdk -> Microsoft.NET.Runtime.Emscripten.Cache?

[joeloff]

@joeloff if emsdk adds a new pack is that going to cause you issues?

nope, should be fine. thanks for checking

[radical]

Does this need to be a separate file?

[lambdageek]

Not sure. I'm copying what the Emscripten.Sdk.pkgproj did

[lambdageek]

I ran the workload build tests in dotnet/runtime with the nugets from this PR (by modifying NuGet.config and eng/Versions.{props,Details.xml} in dotnet/runtme). All the Blazor tests pass.

I also built a multi-threaded runtime and used it with the workload from this PR to build the multithreaded raytracer demo. That also worked.