chore: improve benchmark #8664
Merged
chore: improve benchmark #8664
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Summary of ChangesHello @twoeths, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on improving the efficiency and relevance of the project's benchmark suite. By strategically removing outdated or non-critical tests and optimizing test configurations, the changes aim to reduce overall benchmark execution time, mitigate Out-Of-Memory errors, and ensure that the remaining benchmarks accurately reflect current production performance concerns. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request focuses on improving the performance and relevance of the benchmark suite. The changes effectively reduce benchmark execution time by removing fixed minimum run times (minMs), skipping slower or less relevant test suites, and removing a benchmark file that is no longer needed. Additionally, a minor but useful optimization is included to avoid an unnecessary array copy. The changes are well-aligned with the goal of making the benchmarks faster and more focused. I have one minor suggestion to fix a typo in a comment for better clarity.
Contributor
Performance Report✔️ no performance regression detected Full benchmark results
|
Contributor
Author
|
benchmark execution time was cut down from ~50m to ~13m |
nflaig
reviewed
Dec 3, 2025
| ✔ toHexString serialized data 727592.3 ops/s 1.374396 us/op - 6916 runs 10.0 s | ||
| ✔ Buffer.toString(base64) 2570800 ops/s 388.9840 ns/op - 24628 runs 10.1 s | ||
| */ | ||
| describe("computeSigningRoot", () => { |
Member
There was a problem hiding this comment.
why was this removed instead of just skipped?
Contributor
Author
There was a problem hiding this comment.
as I remember computeSigningRoot() was the bottle neck when we verify attestations
things changed completely since we switch to verifying attestations in batch
it was more about knowing how much does it cause than maintaining its performance over time since the performance implementation is very simple
for clone() and Buffer.from(bytes).toString("base64");: it is what it is, there is nothing to improve there, we also don't want to manage benchmark for them.
nflaig
approved these changes
Dec 3, 2025
wemeetagain
approved these changes
Dec 3, 2025
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #8664 +/- ##
=========================================
Coverage 52.02% 52.02%
=========================================
Files 848 848
Lines 65837 65837
Branches 4811 4811
=========================================
Hits 34250 34250
Misses 31518 31518
Partials 69 69 🚀 New features to boost your workflow:
|
Member
|
🎉 This PR is included in v1.38.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Description
this is based on the long running test I found in https://github.com/ChainSafe/lodestar/actions/runs/19874295397/job/56957698411