QUIC: Read data and get end notification together

[JamesNK]

There doesn't seem to be an API on QuicStream that allows you to read data and get a notification that it is the final chunk of data in one operation.

An API for writing data and ending a stream together exists:

await stream.WriteAsync(data, endStream: true);

But for reading you must always get data and then read again to get a result of 0 which means no more data:

var buffer = GetBuffer();
while (true)
{
    var readCount = await stream.ReadAsync(buffer);
    if (readCount == 0) break;

    // Do stuff with data but don't know whether it is the final data.
}

I think this might be required for Kestrel to properly support HEADERS only requests.

When Kestrel gets the first HEADER frame it needs to start the request. But it also needs to know whether it or not is it ONLY a HEADERS request. If the stream data ends with the HEADERS frame then we process it differently than if the stream is still going and there are DATA frames still to come.

Kestrel can't hang around, waiting on the next QuicStream.ReadAsync to return 0 or a value before it starts the request. That data might be a long time coming (e.g. a long-running gRPC stream where the client doesn't send a message on the stream until 1 minute later)

I think we need ReadAsync to return something like ReadResult. ReadResult has data on it (it could be a data length if you prefer), AND a flag saying whether we are done or not.

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I don't know whether this is important, but there doesn't seem to be an API on QuicStream that allows you to read data and get a notification that it is the final chunk of data in one operation.

An API for writing data and ending a stream together exists:

await stream.WriteAsync(data, endStream: true);

But for reading you must always get data and then read again to get a result of 0 which means no more data:

var buffer = GetBuffer();
while (true)
{
    var readCount = await stream.ReadAsync(buffer);
    if (readCount == 0) break;

    // Do stuff with data but don't know whether it is the final data.
}

Author:	JamesNK
Assignees:	-
Labels:	area-System.IO, untriaged
Milestone:	-

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

There doesn't seem to be an API on QuicStream that allows you to read data and get a notification that it is the final chunk of data in one operation.

An API for writing data and ending a stream together exists:

await stream.WriteAsync(data, endStream: true);

But for reading you must always get data and then read again to get a result of 0 which means no more data:

var buffer = GetBuffer();
while (true)
{
    var readCount = await stream.ReadAsync(buffer);
    if (readCount == 0) break;

    // Do stuff with data but don't know whether it is the final data.
}

I think this might be required for Kestrel to properly support HEADERS only requests.

When Kestrel gets the first HEADER frame it needs to start the request. But it also needs to know whether it or not is it ONLY a HEADERS request. If the stream data ends with the HEADERS frame then we process it differently than if the stream is still going and there are DATA frames still to come.

Kestrel can't hang around, waiting on the next QuicStream.ReadAsync to return 0 or a value because that data might be a long time coming (e.g. a long running gRPC stream where the client doesn't seen a message on the stream until 1 minute later)

Author:	JamesNK
Assignees:	-
Labels:	area-System.Net.Quic, untriaged
Milestone:	-

[JamesNK]

@Tratcher

[geoffkizer]

We could potentially expose an API like this, but I don't think it solves the problem here.

You will still need to handle the case where the headers arrive without a FIN, and then the FIN arrives later.

[Tratcher]

Related discussion: #55347 (comment).

MSQuic looks like it surfaces this state on StreamEventDataReceive.Flags = FIN, QuicStream does not.

[JamesNK]

My knowledge of QUIC isn't strong, but I believe that in QUIC the data and the end stream flag will always arrive together, like how a HEADERS frame in HTTP/2 would have an END_STREAM flag set or not.

STREAM frame? - https://www.rfc-editor.org/rfc/rfc9000.html#section-19.8

What we need with QuicStream is to know whether the data that has come in has had the FIN set. Right now we don't have that capability.

Now, if someone...

1. Decides to send HEADERS in a QUIC frame
2. They intended it to be a HEADERS only request
3. But that QUIC frame with the HEADERS data doesn't report FIN

...then that is a problem with the peer. Kestrel will report an error when it validates the incoming request.

[geoffkizer]

in QUIC the data and the end stream flag will always arrive together

That's not the case. In QUIC, it is perfectly legal to send some data, wait, then send FIN.

like how a HEADERS frame in HTTP/2 would have an END_STREAM flag set or not.

QUIC packets/frames aren't really analogous to HTTP2 frames. They are much more similar to TCP packets.

HTTP3 frames are analogous to HTTP2 frames -- HEADERS, DATA, etc.

Now, if someone...
Decides to send HEADERS in a QUIC frame
They intended it to be a HEADERS only request
But that QUIC frame with the HEADERS data doesn't report FIN
...then that is a problem with the peer. Kestrel will report an error when it validates the incoming request.

Unless there is some particular provision of HTTP3 that outlaws this that I'm not aware of, then this is a perfectly fine thing to do and Kestrel needs to handle it.

Note this is somewhat analogous to sending an HTTP2 HEADERS frame without EndStream, then immediately sending a 0-length DATA frame with the FIN bit set.

[geoffkizer]

They intended it to be a HEADERS only request

BTW, I don't think there is a notion of a "HEADERS only request" in HTTP3, or really in HTTP itself. There are only requests with 0-length bodies.

There are some ways you can detect a 0-length body just from inspecting the headers -- for example, if the Content-Length header is set to 0. But that's not always the case, regardless of version.

[JamesNK]

Unless there is some particular provision of HTTP3 that outlaws this that I'm not aware of, then this is a perfectly fine thing to do and Kestrel needs to handle it.

It is part of the HTTP/3 spec but it is not spelled out obviously. We need it to properly detect and reject certain malformed requests. An example is someone sending a non-zero content-length in a message with no body. We need to reject with an abort, and we need to do it before executing user app code.

https://datatracker.ietf.org/doc/html/draft-ietf-quic-http-34#section-4.1.3

[geoffkizer]

It is part of the HTTP/3 spec but it is not spelled out obviously.

I'm not seeing it here, can you point to what you mean specifically?