Wildcard expansion is silently disabled when a glob expansion encounters an I/O exception

[rainersigwald]

This was reported as a Connect issue.

The repro is simple:

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <MyItem Include="MyDir\**\*.*" Exclude="MyDir\node_modules\**\*.*;MyDir\tmp\**\*.*" />
    </ItemGroup>

    <Target Name="TestInputs">
        <Warning Text="Inputs = @(MyItem)" />
    </Target>
</Project>

Where there are files in MyDir\node_modules that exceed MAX_PATH. That returns:

C:\Users\raines\Downloads>msbuild test.proj
Microsoft (R) Build Engine version 14.0.24720.0
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 12/16/2015 12:46:15 PM.
Project "C:\Users\raines\Downloads\test.proj" on node 1 (default targets).
C:\Users\raines\Downloads\test.proj(8,3): warning : Inputs = MyDir\**\*.*
Done Building Project "C:\Users\raines\Downloads\test.proj" (default targets).

Build succeeded.

Note that the wildcards weren't actually expanded.

The original issue mentions an expectation that the Exclude element would prevent this from happening because that directory and all its contents are excluded. The reason that doesn't work is because we build the Include list fully first, then build the Exclude list, then do a subtraction. The failure occurs when building the Include list, so Exclude has no effect.

[rainersigwald]

There are a couple of questions here.

1. We have (until we figure out MSBuild should handle paths longer than MAX_PATH #53 for full-framework MSBuild) nice errors that we are supposed to emit when something goes over MAX_PATH. Why didn't they fire here?
2. Should we precompute the impact of the exclude list as this user seems to expect?

For 2, I think the answer is "no". Being able to do pattern-based set exclusions seems like an awful lot of work for fairly minimal gain.

[shift-evgeny]

Thanks for troubleshooting this! You're right about my expectations. At minimum MSBuild should report a meaningful error here, but yes, I do expect that excluding the directory will avoid any problems with it.

"Precomputing the impact of the exclude list" is an implementation detail. From the user point of view, the ItemList is "everything in MyDir, except node_modules" - same as manually listing every other sub-directory in the include list (which works).

Yes, it is a problem with NPM that it builds a directory structure that exceeds MAX_PATH, but I have no control over that. All I can do is exclude the directory.

[rainersigwald]

Thinking about it a bit more, it may not be that difficult to consider the Exclude list inline with Include, but I haven't looked at that corner of the code in a while. If it's easy we should do it.

If it's not easy, I think fixing the long-path behavior would be sufficient to avoid this problem--it's inefficient to add and then remove things from a list but would produce the desired behavior.

[rainersigwald]

@shift-evgeny Exclude should now behave as you expect, thanks to improvements made for lazy item evaluation (#770). MSBuild now considers the exclude list while recursing through the include list and will stop when it hits an excluded directory.

This should already be available in VS "15" Preview 5.

I'm not closing this issue because the bad doesn't-expand-wildcards behavior persists if you don't have an exclude. But at least the obvious workaround for node_modules works now.

[Daniel15]

The built-in EmbeddedResource stuff totally breaks because of this because it uses a wildcard like **/*.resx in %ProgramFiles%\Microsoft Visual Studio\2017\community\msbuild\Sdks\Microsoft.NET.Sdk\build\Microsoft.NET.Sdk.DefaultItems.props:

<EmbeddedResource Include="**/*.resx" Exclude="$(DefaultItemExcludes);$(DefaultExcludesInProjectFolder)" Condition=" '$(EnableDefaultEmbeddedResourceItems)' == 'true' " />

Resulting in the GenerateResource task throwing this error:

Microsoft.Common.CurrentVersion.targets(2867,5): error MSB3552: Resource file "**/*.resx" cannot be found.

https://github.com/dotnet/cli/issues/6561

If it's difficult to properly fix this, could you please at least update the resx stuff in Microsoft.Common.CurrentVersion.targets so it doesn't fail due to this problem?

[RehanSaeed]

Just started getting this error from Travis CI here. My path doesn't even seem that long.

/usr/share/dotnet/sdk/1.0.1/Microsoft.Common.CurrentVersion.targets(2865,5): error MSB3552: Resource file "**/*.resx" cannot be found. [/home/travis/build/RehanSaeed/Schema.NET/Source/Schema.NET/Schema.NET.csproj]

I guess my solution is to somehow tell Travis CI to put my code in folder at a lower level?

[Daniel15]

One cause for that error in web apps is node_modules. Older versions of npm had very deep directory structures. If you're using npm, you should ensure that it's the latest version rather than an older version like npm2. You can run "npm install --global npm" to upgrade it. Sent from my phone. On Jul 2, 2017 9:43 AM, "Muhammad Rehan Saeed" <notifications@github.com> wrote: Just started getting this error from Travis CI here <https://travis-ci.org/RehanSaeed/Schema.NET/jobs/249300823>. My path doesn't even seem that long. /usr/share/dotnet/sdk/1.0.1/Microsoft.Common.CurrentVersion.targets(2865,5): error MSB3552: Resource file "**/*.resx" cannot be found. [/home/travis/build/RehanSaeed/Schema.NET/Source/ Schema.NET/Schema.NET.csproj] I guess my solution is to somehow tell Travis CI to put my code in folder at a lower level. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#406 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFnHRWNLx_ydPYoDzxayCOS0jDk8oyIks5sJ8i2gaJpZM4G221q> .

[RehanSaeed]

Not using npm, just a simple class library.