PoV Reclaim (Clawback) Node Side #1462
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
bot fmt |
|
@skunert https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3618768 was started for your command Comment |
|
@skunert Command |
|
I removed usages of the host function in c4b0e05, since it was blocking CI. We use However, I will add the contents of that commit to the follow-up PR. From my point of view this is now ready to merge. I will wait until next week if something comes up in the RFC, otherwise merge. |
bkchr
reviewed
Nov 3, 2023
2 tasks
aramikm
added a commit
to frequency-chain/frequency
that referenced
this pull request
Nov 16, 2023
# Goal The goal of this PR is to evaluate and minimize PoV consumption by `stateful-storage` pallet. Closes #1782 # Discussion - refactored benchmarks to calculate the max of time and PoV - decreased `additional-trie-layers` number - decreased `MaxItemizedPageSizeBytes ` from 64KiB to around 10KiB (will allow around 292 of 32 bytes itemized public keys) # Improvements - by applying mentioned changes the PoV for `apply_item_actions` got reduced from **45KB** to **15KB** - by applying mentioned changes the PoV for `upsert` and `delete` pages got reduced from **12KB** to **6KB** # Future improvements - by using PoV clawback we can further reduce the PoV sizes. A [PR](paritytech/polkadot-sdk#1462) to track # Checklist - [x] Benchmarks added - [x] Weights updated --------- Co-authored-by: Frequency CI [bot] <[email protected]> Co-authored-by: Wil Wade <[email protected]>
paritytech-rfc-bot bot
pushed a commit
to polkadot-fellows/RFCs
that referenced
this pull request
Feb 2, 2024
…lock Utilization (#43) paritytech/polkadot-sdk#1462 has been open for a while and is now well-reviewed. Opening an RFC since it introduces a new host function that is relevant for parachains and light-clients.
|
This pull request has been mentioned on Polkadot Forum. There might be relevant details there: https://forum.polkadot.network/t/parity-tech-update-for-february/6630/1 |
bgallois
pushed a commit
to duniter/duniter-polkadot-sdk
that referenced
this pull request
Mar 25, 2024
This PR provides the infrastructure for the pov-reclaim mechanism discussed in paritytech#209. The goal is to provide the current proof size to the runtime so it can be used to reclaim storage weight. ## New Host Function - A new host function is provided [here](https://github.com/skunert/polkadot-sdk/blob/5b317fda3be205f4136f10d4490387ccd4f9765d/cumulus/primitives/pov-reclaim/src/lib.rs#L23). It returns the size of the current proof size to the runtime. If recording is not enabled, it returns 0. ## Implementation Overview - Implement option to enable proof recording during import in the client. This is currently enabled for `polkadot-parachain`, `parachain-template` and the cumulus test node. - Make the proof recorder ready for no-std. It was previously only enabled for std environments, but we need to record the proof size in `validate_block` too. - Provide a recorder implementation that only the records the size of incoming nodes and does not store the nodes itself. - Fix benchmarks that were broken by async backing changes - Provide new externalities extension that is registered by default if proof recording is enabled. - I think we should discuss the naming, pov-reclaim was more intuitive to me, but we could also go with clawback like in the issue. ## Impact of proof recording during import With proof recording: 6.3058 Kelem/s Without proof recording: 6.3427 Kelem/s The measured impact on the importing performance is quite low on my machine using the block import benchmark. With proof recording I am seeing a performance hit of 0.585%. --------- Co-authored-by: command-bot <> Co-authored-by: Davide Galassi <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
rustadot
added a commit
to rustadot/recurrency
that referenced
this pull request
Sep 5, 2024
# Goal The goal of this PR is to evaluate and minimize PoV consumption by `stateful-storage` pallet. Closes #1782 # Discussion - refactored benchmarks to calculate the max of time and PoV - decreased `additional-trie-layers` number - decreased `MaxItemizedPageSizeBytes ` from 64KiB to around 10KiB (will allow around 292 of 32 bytes itemized public keys) # Improvements - by applying mentioned changes the PoV for `apply_item_actions` got reduced from **45KB** to **15KB** - by applying mentioned changes the PoV for `upsert` and `delete` pages got reduced from **12KB** to **6KB** # Future improvements - by using PoV clawback we can further reduce the PoV sizes. A [PR](paritytech/polkadot-sdk#1462) to track # Checklist - [x] Benchmarks added - [x] Weights updated --------- Co-authored-by: Frequency CI [bot] <[email protected]> Co-authored-by: Wil Wade <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR provides the infrastructure for the pov-reclaim mechanism discussed in #209. The goal is to provide the current proof size to the runtime so it can be used to reclaim storage weight.
New Host Function
Implementation Overview
polkadot-parachain,parachain-templateand the cumulus test node.validate_blocktoo.Impact of proof recording during import
With proof recording: 6.3058 Kelem/s
Without proof recording: 6.3427 Kelem/s
The measured impact on the importing performance is quite low on my machine using the block import benchmark. With proof recording I am seeing a performance hit of 0.585%.