Address key protobuf performance issues

[louiscryan]

This is something of a coverall issue and should be factored out into separate issues for each individual piece of work. Listed here are a number of areas for improving the interaction between protobuf and GRPC to improve overall performance and reduce memory pressure on the system.

In rough order of priority....

• Implement some basic protobuf benchmarks independent of GRPC (scrape the web or extract Googles) as a framework for experimenting with optimizations
• Provide the ability to have a 0-copy approach for writing out large binary sequences (byte buf, file region etc) to the transport without copying
• Provide the ability to read protobufs so that they can be written out again with 0-copy for large sequences. This includes writes to transport and to files
• Document a pattern for transferring a large binary sequence as chunks and utility functions to chunk & dechunk those to application code. File transfer is a common use-case here
• Consider supporting deserialized protobuf backed by pooled buffers (speculative). Would require the notion of a releasable protobuf

@nmittler @ejona86 - Feel free to pile on / cc folks

[louiscryan]

@xfxyjwf FYI

[saintstack]

Yeah, can we have all the above please (smile).

Over in the project I work on (apache hbase):

• The lack of zero-copy is a killer. Give us the rope so we can hang ourselves! We actually rely heavily on this, https://hbase.apache.org/xref/org/apache/hadoop/hbase/util/ByteStringer.html class, where we have hacked java pb so could do some zero-copy (it is wrapped in the ByteStringer class because our hackery has us up against classloader issues when pb jars get loaded by a classloader that is other than that of the applications).
• RPCing, we implement the old pb generated stubs so pb is used to describe which method to invoke as well as the parameters passed back and forth. Because pb'ing all data was untenable -- especially as the size and count of elements goes up -- we hacked up an ugly follow-behind 'sidecar' mechanism so we can pass data without having to envelope it all in pb. If the data is small -- describing coordinates, say -- the pb will just carry the parameter but if large, then the pb notes that there is data following behind and includes meta attributes such as encoding and/or compression. Could the framework do this for us? Its like large binary transfer but not necessarily in chunks (we chunk at a higher level -- not that we'd not appreciate the rpc doing this for us). The zero-copy to the transport would help here too.

Thanks.

[louiscryan]

If we give you 0-copy could the side-car move into the protobuf as a simple binary string? If so then I think we can pretty easily meet your needs

[saintstack]

It could.

We on 2.5 pb currently. It'd require an incompatible upgrade? Thanks Louis.

[louiscryan]

What are your compatability gaps with the proto3 runtime ? (awaits answer in fear)

[saintstack]

What the gap is a TODO.

[tsuna]

The gap is annoying to deal with at best (I tried to make that switch at some point within Arista and we gave up after going deep enough in that rabbit hole and then deciding we had yaks in a more urgent need of a shave).

[louiscryan]

Would a shaded proto3 runtime help ? If you recall where yak shaving was
necessary we can take a look.

On Sun, Oct 11, 2015 at 10:34 PM, Benoit Sigoure notifications@github.com
wrote:

The gap is annoying to deal with at best (I tried to make that switch at
some point within Arista and we gave up after going deep enough in that
rabbit hole and then deciding we had yaks in a more urgent need of a shave).

—
Reply to this email directly or view it on GitHub
#1054 (comment).

[saintstack]

"Would a shaded proto3 runtime help ?"

We could shade, yeah. Would our 2.5 clients be able to talk to a 3.0 server?

Any movement here or something to try?

Our new stumbling block is CodedOutputStream. We have a DirectByteBuffer whose data is offheap and we want it to stay offheap. COS does not support ByteBuffer in 2.5. In 2.6 it does but insists on copying data from the offheap DBB on heap.

We are considering forking PB.

[tsuna]

Forking, ouch. What sorts of arguments does the open-source community need to put forward to convince the protobuf-java maintainers that this issue needs to be tackled upstream?

[nmittler]

@saintstack work is underway to support zero-copy for ByteBuffer. The first part is just adding wrapping support, which is already available on master.

Adding zero-copy support for ByteBuffer to CodedOutputStream is coming soon. Stay tuned.

Agree with @tsuna ... yeah, let's not fork if we can avoid it :)

[saintstack]

That doesn't look bad @nmittler Would it be proto3 only?