Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
9936df1
Add mark&reset methods and canUseByteBuffer&getByteBuffer methods to …
voidzcy Aug 17, 2020
aa93e1d
Default implementations.
voidzcy Aug 17, 2020
454e224
Wire new methods for forwarder
voidzcy Aug 17, 2020
3f47d5e
Support mark&reset and retrieving content via ByteBuffer for netty.
voidzcy Aug 17, 2020
b3c9974
Implementation for composite.
voidzcy Aug 17, 2020
2ede525
Define interface for accessing readable content via ByteBuffers.
voidzcy Aug 17, 2020
3dca312
Implement mark&reset for simple readable buffers.
voidzcy Aug 17, 2020
6249353
Use HasByteBuffer interface for accesing input stream's backing ByteB…
voidzcy Aug 17, 2020
29dce67
Eliminate the length argument for retrieving the ByteBuffer.
voidzcy Aug 17, 2020
22ea6c3
Do no require netty buffer to be direct from API's perspective.
voidzcy Aug 17, 2020
53e347c
Use Deque operations to avoid unncessary moves.
voidzcy Aug 17, 2020
27801fe
Make a list of ByteBuffers up-front instead of a running iterator.
voidzcy Aug 17, 2020
eb71a68
Add getByteBufferSupported method for HasByteBuffer so that it can be…
voidzcy Aug 17, 2020
5d3c657
It's not necessary to implement getByteBuffer for ByteReadbaleBufferW…
voidzcy Aug 17, 2020
9fd8d3c
Add test coverage for mark&reset and getByteBuffer for generic ByteBu…
voidzcy Aug 17, 2020
e3afe50
Add test coverage for netty's special get NIO bytebuffer operation.
voidzcy Aug 17, 2020
e2fdd07
Skip test for operations not supported by okhttp.
voidzcy Aug 17, 2020
033270b
Add test coverage for BufferInputStream with getByteBuffer operation.
voidzcy Aug 17, 2020
0622d51
Add test using a known-length input stream with getByteBuffer operati…
voidzcy Aug 17, 2020
0e8caee
Modify test method name.
voidzcy Aug 17, 2020
ba4e91b
Add test coverage for mark&reset and getByteBuffer for CompositeReada…
voidzcy Aug 17, 2020
69618b2
Add getByteBuffer support for ByteReadableBufferWrapper.
voidzcy Aug 20, 2020
437857d
Only pull ByteBuffers when message is large.
voidzcy Aug 21, 2020
1363505
Run ByteBuffer codepath only in Java 9+.
voidzcy Aug 28, 2020
c46eb73
Slight improvement for avoiding array creation if not necessary.
voidzcy Aug 28, 2020
7b4e070
Merge branch 'master' of github.com:grpc/grpc-java into impl/zero_cop…
voidzcy Aug 28, 2020
772b3ba
Change ReadableBuffer#canUseByteBuffer to hasByteBuffer.
voidzcy Sep 1, 2020
b1c99e5
Removed unnecessary reset.
voidzcy Sep 1, 2020
692076c
Simplify checking runtime java version.
voidzcy Sep 1, 2020
10c13b8
Add ExperimentalApi annotation.
voidzcy Sep 1, 2020
f13c165
Rename ReadableBuffer#hasByteBuffer to getByteBufferSupported.
voidzcy Sep 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add getByteBufferSupported method for HasByteBuffer so that it can be…
… used for okhttp as well.
  • Loading branch information
voidzcy committed Aug 17, 2020
commit eb71a68dfb6697369d99df68b756b0f3e0909280
10 changes: 9 additions & 1 deletion api/src/main/java/io/grpc/HasByteBuffer.java
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,21 @@
// TODO(chengyuanzhang): add ExperimentalApi annotation.
public interface HasByteBuffer {

/**
* Indicates whether or not {@link #getByteBuffer} operation is supported.
*/
boolean getByteBufferSupported();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not give this the same name as the equivalent added ReadableBuffer method (my suggestion would be hasByteBuffer())?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to match what we use for ReadableBuffer right? We'd probably have a discussion on this API this Thursday and we usually do a vote for the name.

hasByteBuffer 1 vote now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As well as for consistency, having it line up with ReadableBuffer would permit ReadableBuffers to themselves be InputStreams implementing this interface, which I think might allow for some other streamlining later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, for that reason.

My original thought was this HasByteBuffer interface should only have the getByteBuffer method. InputStreams not supporting the operation should just not implement this interface. #7330 (comment) suggests combining two InputStream implementations into one by adding this getByteBufferSupported method. This look ok, but making the method hasByteBuffer does make this API wired. I'd rather change the hasByteBuffer method on ReadableBuffer interface to getByteBufferSupported.


/**
* Gets a {@link ByteBuffer} containing some bytes of the content next to be read, or {@code
* null} if has reached end of the content. The number of bytes contained in the returned buffer
* is implementation specific. Calling this method does not change the position of the input
* stream. The returned buffer's content should not be modified, but the position, limit, and
* mark may be changed. Operations for changing the position, limit, and mark of the returned
* buffer does not affect the position, limit, and mark of this input stream.
* buffer does not affect the position, limit, and mark of this input stream. This is an optional
* method, so callers should first check {@link #getByteBufferSupported}.
*
* @throws UnsupportedOperationException if this operation is not supported.
*/
@Nullable
ByteBuffer getByteBuffer();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make the API cleaner how about having this return null for not supported and a (constant) empty ByteBuffer for EOS?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having this return null for not supported and a (constant) empty ByteBuffer for EOS

The semantics of returning null for operation not supported isn't strong enough. Most existing Java APIs (e.g.,java.nio.ByteBuffer uses hasArray() and array()) throw an UnsupportedOperationException for unsupported optional APIs. I think the current approach is cleaner.

Expand Down
29 changes: 10 additions & 19 deletions core/src/main/java/io/grpc/internal/ReadableBuffers.java
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,7 @@ public static String readAsStringUtf8(ReadableBuffer buffer) {
* @param owner if {@code true}, the returned stream will close the buffer when closed.
*/
public static InputStream openStream(ReadableBuffer buffer, boolean owner) {
if (!owner) {
buffer = ignoreClose(buffer);
}
return buffer.canUseByteBuffer()
? new ByteBufferInputStream(buffer) : new BufferInputStream(buffer);
return new BufferInputStream(owner ? buffer : ignoreClose(buffer));
}

/**
Expand Down Expand Up @@ -334,7 +330,8 @@ public ByteBuffer getByteBuffer() {
/**
* An {@link InputStream} that is backed by a {@link ReadableBuffer}.
*/
private static class BufferInputStream extends InputStream implements KnownLength {
private static final class BufferInputStream extends InputStream
implements KnownLength, HasByteBuffer {
final ReadableBuffer buffer;

public BufferInputStream(ReadableBuffer buffer) {
Expand Down Expand Up @@ -390,26 +387,20 @@ public boolean markSupported() {
}

@Override
public void close() throws IOException {
buffer.close();
}
}

/**
* A {@link ReadableBuffer}-backed {@link InputStream} that supports the operation of getting
* bytes via {@link ByteBuffer}s.
*/
private static class ByteBufferInputStream extends BufferInputStream implements HasByteBuffer {

ByteBufferInputStream(ReadableBuffer buffer) {
super(buffer);
public boolean getByteBufferSupported() {
return buffer.canUseByteBuffer();
}

@Nullable
@Override
public ByteBuffer getByteBuffer() {
return buffer.getByteBuffer();
}

@Override
public void close() throws IOException {
buffer.close();
}
}

private ReadableBuffers() {}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ public T parse(InputStream stream) {
int size = stream.available();
// TODO(chengyuanzhang): we may still want to go with the byte array approach for small
// messages.
if (stream instanceof HasByteBuffer && stream.markSupported()) {
if (stream instanceof HasByteBuffer
&& stream.markSupported() && ((HasByteBuffer) stream).getByteBufferSupported()) {
List<ByteBuffer> buffers = new ArrayList<>();
stream.mark(size);
while (stream.available() != 0) {
Expand Down