feat: transfer pending gossipsub message msg data #8689
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Summary of ChangesHello @twoeths, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant memory optimization for handling Gossipsub messages by leveraging Node.js Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a memory optimization for handling gossipsub messages between the network and main threads. It achieves this by using the transferList option when posting messages, which transfers ownership of the message data buffer instead of copying it. This is a solid approach to reduce memory overhead. Additionally, the EventDirection enum has been refactored to a more appropriate location, improving code organization. The changes look good, and I have one suggestion to improve the type safety and clarity of the new logic for transferring buffers.
Contributor
Performance Report🚀🚀 Significant benchmark improvement detected
Full benchmark results
|
twoeths
commented
Dec 12, 2025
| data, | ||
| }; | ||
| parentPort.postMessage(workerEvent); | ||
| let transferList: ArrayBuffer[] | undefined = undefined; |
Contributor
Author
There was a problem hiding this comment.
this is the only change
others were to avoid cyclic dependencies
nflaig
approved these changes
Dec 16, 2025
wemeetagain
approved these changes
Dec 16, 2025
wemeetagain
deleted the
te/transfer_pending_gossipsub_message_msg_data
branch
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #8689 +/- ##
============================================
- Coverage 52.04% 52.04% -0.01%
============================================
Files 848 848
Lines 65734 65727 -7
Branches 4807 4806 -1
============================================
- Hits 34214 34208 -6
+ Misses 31451 31450 -1
Partials 69 69 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Buffer.alloc()instead ofBuffer.allocUnsafe(). We don't have to feel bad about that becauseBuffer.allocUnsafe()does not work with this PR, and we don't waste any memory.Description
transferListparam when posting messages from network thread to the main threadpart of #8629
Testing
I've tested this on
feat2for 3 days, the previous branch was #8671 so it's basically the current stable, does not see significant improvement but some good data for different nodesnovcsasnode we have better memory there on main thread with same mesh peers, same memory on network threadsasnode, we have better memory on network thread, a little bit worse on the main threadforward msg avg peers, we're faster than majority of them