[release/2.2] Change HttpConnectionPool.GetConnectionAsync to do read-ahead outside of lock (#)

[stephentoub]

Port #32495 to release/2.2.

Minimize the work done inside the lock in order to reduce contention. This means we need to reacquire the lock if the connection isn't usable, but in the fast path case where the connection is usable, we remove the syscalls from being performed while holding the lock.

Related to https://github.com/dotnet/corefx/issues/31799

[stephentoub]

Description

SocketsHttpHandler pools connections, and when it takes a connection from the pool, it checks to see whether the connection is still alive. That check is currently happening while a lock is held, which under heavy load and many short requests can result increased contention on the lock. This change moves that check to outside of the lock, so that in the fast path / common case, the lock is held for a very short period of time.

Customer Impact

For microservices where many ASP.NET requests in turn make many HttpClient calls to other services, the contention can result in significantly reduced throughput than is otherwise possible. The benchmark results shown in #32495 (comment) highlight this impact.

Regression?

No

Risk

This code path is critical to SocketsHttpHandler, used in the processing of all requests, so a mistake here could be impactful. But the change itself is minimal: it moves some checks from inside the lock to outside the lock. The net result is that for the common case, we still take the lock once, but we hold it for a much shorter period of time. In the less common case where we were able to grab a connection but then discover after releasing the lock and doing these checks that the connection is no longer valid (e.g. the server closed it after a timeout), we end up needing to loop around and take the lock again, for a small increase in cost on the uncommon path.

[Eilon]

I approve for 2.2, though I don't know the code.

[stephentoub]

The Windows failure is an unrelated pipelines test: ReadAsyncCompletesIfFlushAsyncCanceledMidFlush.

@dotnet-bot test Outerloop Windows x64 Debug Build please

[stephentoub]

cc: @geoffkizer

[jamshedd]

Approved for 2.2.

[stephentoub]

The Outerloop Windows failures are unrelated:

• System.MemoryTests.MemoryTests.MemoryManagerPinLargeArray, failed with an OOM on Win10
• GetResponseAsync_ServerNameNotInDns_ThrowsWebException, due to https://github.com/dotnet/corefx/issues/31147 which was fixed in master but not other branches

[danmoseley]

@stephentoub you can hit merge when CI is green (or effectively so)

[stephentoub]

Thanks