Child channels can not access their own peerMaxMessageSize

[gwynne]

There does not appear to be any way for a child channel implementation to get access to the peerMaxMessageSize value derived from the SSH_MESSAGE_CHANNEL_OPEN message. This value is needed by some subsystems, such as SFTP, which are required to manually perform splitting of large data packets manually. In SFTP's case, offering an oversized value in a WriteFile request causes some servers to enter a wedged state waiting for data that will never arrive; without access to the channel's desired maximum message size, it must assume the worst (the specification-required minimum supported size of 32768 payload bytes), which is obviously less than ideal.

[Lukasa]

Can you clarify the specific issue? swift-nio-ssh will manually fragment channel data messages: is there some reason to need to fragment beyond that?

If there is, the best thing to do is to add a new ChannelOption that can be used to retrieve the value.

[gwynne]

SFTP file transfers have to be manually chunked by max(32768, peerMaxMessageSize) bytes per explicit SSH_FXP_WRITE packet.

The SFTP packet format includes a leading data length value which is required to be less than or equal to the max message size of the child channel (with the spec explicitly reinforcing the SSH transport layer's minimum requirement of 32768 payload bytes). The resulting packet may be fragmented by the SSH window flow control, but the total size of all fragments must still be within the max message size - it's arguably incorrect behavior for NIOSSH to accept and fragment a packet that exceeds the maximum packet size of the channel involved (the windowing mechanism being per-channel anyhow), though I know not all subsystems are anywhere near as picky as SFTP seems to be about it.

This being said, most of the responsibility is on the shoulders of the SFTP server - servers in the wild are apparently inconsistent at best about validating that an incoming packet is of valid size. In at least one case (the one which inspired this issue in the first place), a particularly lax implementation:

• Does not validate the size of the packet
• Does not correctly reassemble fragments of an oversized packet after failing to complain about its size
• Either 1) silently drops all fragments which contain data past the "maximum size" boundary, or 2) incorrectly treats those fragments as new packets and then fails to report an invalid packet error.
• Does not consider the packet "complete" at the transport layer despite having received all of the fragments (and mishandled them), meaning no response (success or failure) is sent and no further commands sent over the connection are processed.
• Does treat the connection as alive and "active" while "waiting" for data; neither the server nor most client implementations will ever time out (neither the SSH connection itself nor the child channel) if the connection isn't otherwise shut down.

Obviously this is a particularly badly-behaved server, but it nonetheless requires the SFTP client implementation to know the maximum packet size so that it can correctly split the data into multiple WriteFile packets. The client can (and has to, at present) assume the required minimum maximum packet size (yes.) of 32768 bytes, but obviously that's a bit of an efficiency issue if, for example, the server had just said "hey you can send me 1048576 bytes at a time!", especially given that (surprisingly enough) SFTP actually supports 64-bit file offsets and thus can handle files larger than 4GB.

As far as I can figure out, the "rule" NIOSSH isn't enforcing as far as fragmenting goes would be something like: "Fragmenting a ChannelDataMessage must only occur in response to insufficient space available in the channel window, not as a consequence of exceeding the maximum packet size." - or in other words, fragmenting is for backpressure, not for implementing a stream protocol over a datagram protocol. At least part of the problem seems to be that many servers and subsystems accept oversize packets without issue while others misbehave spectacularly. Another is that the SSH connection and transport layer specifications IMO go out of their way to be unclear about this 😕.

I'll see about opening a PR to add a channel option 🙂.

[gwynne]

(P.S.: Yes, I did admit somewhere in the middle there that the max packet size value isn't so much "required" as it would improve efficiency 🤣. It'd also be pointless in the case of the specific server I described - the maximum packet size it specifies is exactly the SSH-required minimum value anyway 😅.)