This repository was archived by the owner on Jan 22, 2025. It is now read-only.
Enable conservative out-of-bound snapshot cleaning #8811
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ryoqun
commented
Mar 12, 2020
| } | ||
|
|
||
| #[test] | ||
| fn test_accounts_purge_chained_purge_after_snapshot_restore_complex() { |
Contributor
Author
There was a problem hiding this comment.
I understood this mystical test finally...
Contributor
Author
There was a problem hiding this comment.
Moreover, https://github.com/solana-labs/solana/pull/8811/files#diff-2099c5256db4eb5975c8834af38f6456R1497 made this previous form of this test fail.
ryoqun
force-pushed
the
conservative-outbound-snapshot-cleaning
branch
from
d372bdd to
4cc6cee
Compare
Codecov Report
@@ Coverage Diff @@
## master #8811 +/- ##
======================================
Coverage 80.0% 80.0%
======================================
Files 265 265
Lines 57477 57481 +4
======================================
+ Hits 45998 46011 +13
+ Misses 11479 11470 -9 |
Contributor
Author
|
I ran v1.0 validator with this PR patched overnight, and it seems that it still produces valid snapshots. Overall, size reduction is 10x and restore-time reduction is 20x. That's amazing. |
Contributor
Author
|
Also, snapshot creation-time reduction is 20x (10min -> 30s): grafana-testnet-monitor-edge-ryoqun-without-out-of-bound-snaphost-cleaning.pdf |
Contributor
|
That is amazing!! |
mergify bot
pushed a commit
that referenced
this pull request
Mar 13, 2020
* Enable conservative out-of-bound snapshot cleaning * Add tests (cherry picked from commit 4bbf09f)
solana-grimes
pushed a commit
that referenced
this pull request
Mar 13, 2020
Contributor
Author
|
I also tested against the SLP cluster. And there was no problem, too. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
There is no codepath for cleaning unused storages in snapshot when starting a validator
Summary of Changes
Our past efforts has paid off; Now just need small plumbing and test clarification.
And, I'm now very conservative with good reason of newly-introduced bugs with the adventure of #8168. So, this PR doesn't contain aggressive strategy, originally had in #8337.
Let it rip little by little. :)
Not codewise, but this PR requires #8724
Split from #8337
Towards #8168