Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Rework ExclusiveAddressUse and ReuseAddress on non-Windows platforms
The ExclusiveAddressUse socket option is a Windows-specific option that, when set to "true," tells Windows not to allow another socket to use the same local address as this socket.  This is only needed on Windows because it otherwise (at least in some versions/configurations) allows any socket with the ReuseAddress option set to "steal" the address from a socket that did not have *any* options set.

On Unix, we previously treated this as an "unsupported" option.  However, it is recommended to set this option to "true" on Windows, to avoid malicious theft of a service's address, so we need to support the option, in some fashion, on Unix, so that it's possible to write portable code that works reliably everywhere.  Since the *only* behavior on Linux/OSX is equivalent to "ExclusiveAddressUse=true" on Windows, we implement this option as a no-op if it's set to "true," and as an unsupported option if set to "false."  (It's possible that we could come up with a better failure for the "false" case, but I'm treating it as "unsupported" for compatiblity with the 1.0 release).

Another related option is ReuseAddress.  On Windows, this option allows a socket's address *and* port to be reused.  It's equivalent to *two* native options on Unix: SO_REUSEADDR and SO_REUSEPORT.  Again, for portability, we need an option that will work roughly the same way on all platforms.  We could introduce a new option (ReuseAddressAndPort?) but existing code is already using the current ReuseAddress option.  So this change makes ReuseAddress set both SO_REUSEADDR and SO_REUSEPORT on Unix.  If we need to support these options individually, on Unix only, in the future, we'll need to introduce two new options (maybe ReuseAddressOnly and ReusePort) which will likely need to be treated as "unsupported" options on Windows.  For now, no need for managed support for more fine-grained options has been demonstrated.

For more information on the underlying native options on Windows, Linux, OSX, etc., see this great writeup [on stackoverflow](http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t).  Also, the Windows docs [discuss the SO_EXCLUSIVEADDR option](https://msdn.microsoft.com/en-us/library/windows/desktop/cc150667(v=vs.85).aspx) in depth.
  • Loading branch information
ericeil authored and Eric Eilebrecht committed Sep 22, 2016
commit dca9ceb5cfce6aac822012b0a68648f075e2ea98
86 changes: 86 additions & 0 deletions src/Native/Unix/System.Native/pal_networking.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2071,6 +2071,51 @@ extern "C" Error SystemNative_GetSockOpt(

int fd = ToFileDescriptor(socket);

//
// Handle some special cases for compatibility with Windows
//
if (socketOptionLevel == PAL_SOL_SOCKET)
{
if (socketOptionName == PAL_SO_EXCLUSIVEADDRUSE)
{
//
// SO_EXCLUSIVEADDRUSE makes Windows behave like Unix platforms do WRT the SO_REUSEADDR option.
// So, for non-Windows platforms, we act as if SO_EXCLUSIVEADDRUSE is always enabled.
//
if (*optionLen != sizeof(int32_t))
{
return PAL_EINVAL;
}

*reinterpret_cast<int32_t*>(optionValue) = 1;
return PAL_SUCCESS;
}
else if (socketOptionName == PAL_SO_REUSEADDR)
{
//
// On Windows, SO_REUSEADDR allows the address *and* port to be reused. It's equivalent to
// SO_REUSEADDR + SO_REUSEPORT other systems. Se we only return "true" if both of those options are true.
//
auto optLen = static_cast<socklen_t>(*optionLen);

int err = getsockopt(fd, SOL_SOCKET, SO_REUSEADDR, optionValue, &optLen);

if (err == 0 && *reinterpret_cast<uint32_t*>(optionValue) != 0)
{
err = getsockopt(fd, SOL_SOCKET, SO_REUSEPORT, optionValue, &optLen);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As pointed out in the design discussion, this could be confusing: what if on *NIX I only want to set SO_REUSEPORT but not SO_REUSEADDR?
Is that a a valid *NIX scenario? If yes, is there a way to achieve that somehow?

Copy link
Contributor Author

@ericeil ericeil Sep 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a a valid *NIX scenario? If yes, is there a way to achieve that somehow?

I'm not sure if that's a valid scenario. It's not one anyone's asking for right now. 😃

The premise of this change is that we have an already-established semantic for ReuseAddress in managed code, and there's more value in this semantic being portable than having the managed socket option names map 1:1 with native socket options. I agree that breaking that 1:1 mapping may be confusing for folks who are specifically targeting Unix socket options, but keeping the 1:1 mapping is also confusing, to those who are trying to get consistent behavior across platforms.

Copy link

@christianhuening christianhuening Sep 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I've done a bit of research and found that SO_REUSEPORT seems to be a bit exotic yet useful in extreme load scenarios (i.e. at Google: https://lwn.net/Articles/542629/).
My guess is that no one would ask for this in the near future, since it hasn't been available in .NET Framework so far. However can it hurt to add an option to allow to possibly tweak high performance socket code in special conditions, when you know you're running on *NIX?

EDIT: Here is the original motivation for this feature (source: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c617f398edd4db2b8567a28e899a88f8f574798d):

The motivating case for so_resuseport in TCP would be something like
a web server binding to port 80 running with multiple threads, where
each thread might have it's own listener socket. This could be done
as an alternative to other models: 1) have one listener thread which
dispatches completed connections to workers. 2) accept on a single
listener socket from multiple threads. In case #1 the listener thread
can easily become the bottleneck with high connection turn-over rate.
In case #2, the proportion of connections accepted per thread tends
to be uneven under high connection load (assuming simple event loop:
while (1) { accept(); process() }, wakeup does not promote fairness
among the sockets. We have seen the disproportion to be as high
as 3:1 ratio between thread accepting most connections and the one
accepting the fewest. With so_reusport the distribution is
uniform.

Nginx uses it as well: https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/
And Apache does so too: https://httpd.apache.org/docs/trunk/en/mod/mpm_common.html#listencoresbucketsratio

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @christianhuening for the investigation!

Maybe properly supporting this and documenting/adding a new property/method would be a better long-term plan.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However can it hurt to add an option to allow to possibly tweak high performance socket code in special conditions, when you know you're running on *NIX?

Once Socket has its public Handle property added back, this could be done via a P/Invoke that's only invoked when on Unix. That could serve as a stop-gap until such time as it was determined to be important to add .NET APIs for.

}

if (err != 0)
{
return SystemNative_ConvertErrorPlatformToPal(errno);
}

assert(optLen <= static_cast<socklen_t>(*optionLen));
*optionLen = static_cast<int32_t>(optLen);
return PAL_SUCCESS;
}
}

int optLevel, optName;
if (!TryGetPlatformSocketOption(socketOptionLevel, socketOptionName, optLevel, optName))
{
Expand Down Expand Up @@ -2099,6 +2144,47 @@ SystemNative_SetSockOpt(intptr_t socket, int32_t socketOptionLevel, int32_t sock

int fd = ToFileDescriptor(socket);

//
// Handle some special cases for compatibility with Windows
//
if (socketOptionLevel == PAL_SOL_SOCKET)
{
if (socketOptionName == PAL_SO_EXCLUSIVEADDRUSE)
{
//
// SO_EXCLUSIVEADDRUSE makes Windows behave like Unix platforms do WRT the SO_REUSEADDR option.
// So, on Unix platforms, we consider SO_EXCLUSIVEADDRUSE to always be set. We allow manually setting this
// to "true", but not "false."
//
if (optionLen != sizeof(int32_t))
{
return PAL_EINVAL;
}

if (*reinterpret_cast<int32_t*>(optionValue) == 0)
{
return PAL_ENOTSUP;
}
else
{
return PAL_SUCCESS;
}
}
else if (socketOptionName == PAL_SO_REUSEADDR)
{
//
// On Windows, SO_REUSEADDR allows the address *and* port to be reused. It's equivalent to
// SO_REUSEADDR + SO_REUSEPORT other systems.
//
int err = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, optionValue, static_cast<socklen_t>(optionLen));
if (err != 0)
{
err = setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, optionValue, static_cast<socklen_t>(optionLen));
}
return err == 0 ? PAL_SUCCESS : SystemNative_ConvertErrorPlatformToPal(errno);
}
}

int optLevel, optName;
if (!TryGetPlatformSocketOption(socketOptionLevel, socketOptionName, optLevel, optName))
{
Expand Down
2 changes: 1 addition & 1 deletion src/Native/Unix/System.Native/pal_networking.h
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ enum SocketOptionName : int32_t
PAL_SO_LINGER = 0x0080,
PAL_SO_OOBINLINE = 0x0100,
// PAL_SO_DONTLINGER = ~PAL_SO_LINGER,
// PAL_SO_EXCLUSIVEADDRUSE = ~PAL_SO_REUSEADDR,
PAL_SO_EXCLUSIVEADDRUSE = ~PAL_SO_REUSEADDR,
PAL_SO_SNDBUF = 0x1001,
PAL_SO_RCVBUF = 0x1002,
PAL_SO_SNDLOWAT = 0x1003,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -233,5 +233,65 @@ private static Socket CreateBoundUdpSocket(out int localPort)
localPort = (receiveSocket.LocalEndPoint as IPEndPoint).Port;
return receiveSocket;
}

[Theory]
[InlineData(null, null, null, true)]
[InlineData(null, null, false, true)]
[InlineData(null, false, false, true)]
[InlineData(null, true, false, true)]
[InlineData(null, true, true, false)]
[InlineData(true, null, null, true)]
[InlineData(true, null, false, true)]
[InlineData(true, null, true, true)]
[InlineData(true, false, null, true)]
[InlineData(true, false, false, true)]
[InlineData(true, false, true, true)]
public void ReuseAddress(bool? exclusiveAddressUse, bool? firstSocketReuseAddress, bool? secondSocketReuseAddress, bool expectFailure)
{
using (Socket a = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
if (exclusiveAddressUse.HasValue)
{
a.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ExclusiveAddressUse, exclusiveAddressUse.Value);
}
if (firstSocketReuseAddress.HasValue)
{
a.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, firstSocketReuseAddress.Value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have existing tests verifying the roundtripping behavior for SocketOptionName.ReuseAddress with GetSocketOption?

}

a.Bind(new IPEndPoint(IPAddress.Loopback, 0));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any need to test both IPv4 and IPv6, or is just IPv4 sufficient for this purpose?


using (Socket b = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
if (secondSocketReuseAddress.HasValue)
{
b.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, secondSocketReuseAddress.Value);
}

if (expectFailure)
{
Assert.ThrowsAny<SocketException>(() => b.Bind(a.LocalEndPoint));
}
else
{
b.Bind(a.LocalEndPoint);
}
}
}
}

[Theory]
[PlatformSpecific(PlatformID.Windows)]
[InlineData(false, null, null, true)]
[InlineData(false, null, false, true)]
[InlineData(false, false, null, true)]
[InlineData(false, false, false, true)]
[InlineData(false, true, null, true)]
[InlineData(false, true, false, true)]
[InlineData(false, true, true, false)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the expected behavior of these cases on Unix? Should we verify those failures via a test that's Unix-only?

public void ReuseAddress_Windows(bool? exclusiveAddressUse, bool? firstSocketReuseAddress, bool? secondSocketReuseAddress, bool expectFailure)
{
ReuseAddress(exclusiveAddressUse, firstSocketReuseAddress, secondSocketReuseAddress, expectFailure);
}
}
}
31 changes: 25 additions & 6 deletions src/System.Net.Sockets/tests/FunctionalTests/TcpClientTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ public void ConnectedAvailable_NullClient()
[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void ExclusiveAddressUse_NullClient()
public void ExclusiveAddressUse_NullClient_Windows()
{
using (TcpClient client = new TcpClient())
{
Expand All @@ -100,13 +100,33 @@ public void ExclusiveAddressUse_NullClient()

[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet()
[PlatformSpecific(~PlatformID.Windows)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we've used PlatformID.AnyUnix for this rather than ~Windows.

public void ExclusiveAddressUse_NullClient_NonWindows()
{
using (TcpClient client = new TcpClient())
{
client.Client = null;

Assert.True(client.ExclusiveAddressUse);
}
}

[Fact]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet_True()
{
using (TcpClient client = new TcpClient())
{
client.ExclusiveAddressUse = true;
Assert.True(client.ExclusiveAddressUse);
}
}

[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet_False()
{
using (TcpClient client = new TcpClient())
{
client.ExclusiveAddressUse = false;
Assert.False(client.ExclusiveAddressUse);
}
Expand All @@ -115,14 +135,13 @@ public void Roundtrip_ExclusiveAddressUse_GetEqualsSet()
[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.AnyUnix)]
public void ExclusiveAddressUse_NotSupported()
public void ExclusiveAddressUse_Set_False_NotSupported()
{
using (TcpClient client = new TcpClient())
{
Assert.Throws<SocketException>(() => client.ExclusiveAddressUse);
Assert.Throws<SocketException>(() =>
{
client.ExclusiveAddressUse = true;
client.ExclusiveAddressUse = false;
});
}
}
Expand Down