-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Multi tree support #3386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cheme
wants to merge
169
commits into
paritytech:master
Choose a base branch
from
cheme:cheme/new-trie2
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Multi tree support #3386
Changes from 1 commit
Commits
Show all changes
169 commits
Select commit
Hold shift + click to select a range
a70794d
Use the new trie
arkpar f544f79
Fixes for parity-db
arkpar c135b9f
Merge branch 'master' of github.com:paritytech/polkadot-sdk
arkpar c6fca87
Merge branch 'master' of github.com:paritytech/polkadot-sdk into arkp…
arkpar 3d1b7b5
patch
cheme 2859b82
location trait
cheme 7a984c6
pass location to trie for child usage
cheme 09f4f09
change to child trie access (state machine and trie crates)
cheme f7fb149
code change for count state mig
cheme 8ff6c2a
pass child changeset in main trie one
cheme 024889a
flatten trie_commit
cheme dde33fc
adapt, not right db branch.
cheme 6c66129
right parity-db branch and remaining fixes
cheme a85164d
others
cheme 866ddc6
update for deps
cheme c432446
trie root rem
cheme 69a6020
adapt, tests sc-client-db full ko
cheme 90b753d
sc-client-db tests
cheme 532e352
child changeset single alias
cheme 0b0a428
update patches
cheme 8716a26
merge will need work/rework
cheme 9ba6a2d
resolve sp_trie merge (state machine will need recorder rework next)
cheme c39a35f
Use MemTreeDB on some sp_trie tests.
cheme 32e75ff
actually keyspace the data in changeset
cheme d686d50
rename to avoid confusion with other with_recorder
cheme 1653b67
wrap should not be usefull if using with_temp_recorder.
cheme a980c65
memo
cheme 1f878a9
restore prefix memory db by default
cheme f196361
fix test
cheme 930b0b0
sp-api, here hard coded location do not impact struct sizes.
cheme f676126
fix client db merge, less recheck
cheme f19cca4
need the lock around recorder.
cheme 45df4c8
bad direction
cheme 3519e36
Revert "bad direction"
cheme 9cdc9b1
dirty solution, maybe move back to warp code but using an Arc on top of
cheme d7b1f01
a bit more, bef trying warp
cheme 52242a1
fix a bit
cheme e097d07
Fixes.
cheme 2644056
naive option def
cheme f89b2c0
pass multitree to db init
cheme 5d98d30
options for multitree
cheme 09f658f
somehow aligned...
cheme 76c7451
Merge branch 'master' into tt
cheme 8a4f0fa
test
cheme b5bd725
resolving
cheme caeb0fa
Rem standardmap
cheme 94ef589
fmts
cheme 6a1e031
fmt2
cheme b4e8e36
benches
cheme eacd0d8
few clippy warn
cheme 84c5375
fmt
cheme 9a317be
fix
cheme 70cc271
fix test
cheme 5e0f0a2
allow multi tree direct access
cheme bffb94b
store is multi tree in db adapter
cheme 9a28dd4
restore key stripping for rocksdb ref count no multitree
cheme f6b23aa
purge chain test over multi
cheme 7a7997a
missing i
cheme 958b51b
Merge branch 'master' into cheme/new-trie2
cheme 742f008
fix
cheme 056231f
change to use prefix on contains root
cheme 1a04af6
Revert "change to use prefix on contains root"
cheme b9ba5e4
use empty prefix on contains root
cheme d58d72a
apply_to with no prefix
cheme 1748085
missing
cheme 8af3690
fmt
cheme 8a87c80
test
cheme f786dd6
proper test
cheme 584c159
clipy
cheme 1f3e031
restore (issue may just be to not use empty for next values when old
cheme 70666d0
Merge branch 'master' into cheme/new-trie2
cheme 0fdeef2
to check
cheme 83182d0
workable tests TODO revert and move to bench
cheme c7311b6
state_capabilities of db
cheme 5a9e158
initial bench feed
cheme 893a29a
append only should not ref count
cheme c457b5f
avoid migrating existing db
cheme 54c447e
fmt
cheme 6281aad
Do not assume no statedb is multitree (eg benchmark)
cheme 2bfb662
contains with get
cheme 9c0e59a
fix benchmarks
cheme fb0bf23
fmt
cheme 20cb03f
revert renaming to TrieCommit.
cheme bc5c377
Merge branch 'master' into cheme/new-trie2
cheme f5577fc
Merge branch 'master' into cheme/new-trie2
cheme d987a01
fix
cheme c2a1f30
fix
cheme 9cde418
fix sp-state-machine
cheme cc69d31
unused import
cheme f47f8bd
rem checks
cheme e29923c
non needed std
cheme 0892234
missing
cheme e1a77e2
actually need std (maybe refcell the no std)
cheme 2e553be
allow dead code, should really consider implement with refcell
cheme e7c9525
add with recorder in no std
cheme c0c3b7f
send sync unsafe impl (should only be for wasm)
cheme d3f62ba
real unsafe
cheme 410c8d1
other
cheme c97c77d
switch to subtrie
cheme 1b30649
Merge branch 'master' into cheme/new-trie2
cheme b8f47b6
Merge branch 'master' into cheme/new-trie2
cheme d600175
switch
cheme 425120f
update
cheme eeeee67
small fix and trace
cheme 4ca70b7
update
cheme f0ec96e
f
cheme 1d70579
Merge branch 'master' into cheme/new-trie2
cheme ed73a60
fix
cheme 794b4a8
fmt
cheme 35d913e
fix old write storage bench
cheme 7a22d90
fix multi (slow)
cheme 06e32c9
mix node conf multi and rocks
cheme 449f6ae
fmt
cheme e5ec853
missing
cheme a769dcc
Use location internally a bit more.
cheme af068fb
rem some todos
cheme ac74627
partially restore test
cheme e375922
try restoring clone (may be an issue)
cheme 6381c21
bench state access
cheme bfb7aa5
benches
cheme 9d97920
update
cheme b8aa260
clippy to maste
cheme 5848ecb
Merge branch 'master' into cheme/new-trie2
cheme 89a72b3
fmt
cheme def9b7c
fix merge
cheme d095643
fmt
cheme 1bba97f
fix
cheme 042288d
fix fix
cheme adff644
fix
cheme d41a02a
Merge
MattHalpinParity 1e7145e
Merge master
MattHalpinParity 2178de5
Merge master
MattHalpinParity a014145
Merge master
MattHalpinParity eb9322b
Merge master
MattHalpinParity 51b5c6f
Merge master
MattHalpinParity 4a7a2bd
Merge master
MattHalpinParity 73441a8
Workspace dependencies
MattHalpinParity 668703c
Remove unused workspace dependencies
MattHalpinParity b50970b
Merge master
MattHalpinParity 9f7ff78
Merge master
MattHalpinParity 9cd9290
Merge master
MattHalpinParity 186846c
Merge master
MattHalpinParity 2565390
Merge master
MattHalpinParity e03fcc3
Merge master
MattHalpinParity e2c91e5
Merge master
MattHalpinParity 780f63f
Merge branch master into cheme/new-trie2
MattHalpinParity eeeac52
Merge branch master into cheme/new-trie2
MattHalpinParity 14b3286
Merge branch master into cheme/new-trie2
MattHalpinParity 10d729b
Fixes
MattHalpinParity 1f7e810
Fixes
MattHalpinParity 1f59d23
Fixes and fmt
MattHalpinParity 2afead2
parity-db patch no longer needed
MattHalpinParity ce2fcdc
Merge branch master into cheme/new-trie2
MattHalpinParity 105aff1
Merge branch master into cheme/new-trie2
MattHalpinParity bd3c214
Merge branch master into cheme/new-trie2
MattHalpinParity 216f619
Compile fixes
MattHalpinParity 96d75b5
fmt
MattHalpinParity 8185962
taplo format
MattHalpinParity 8c5b5ba
subtrie patch no longer needed
MattHalpinParity 854af66
Merge branch master into cheme/new-trie2
MattHalpinParity e41f92c
Merge branch master into cheme/new-trie2
MattHalpinParity 4e89372
prdoc 3386
MattHalpinParity 24ee2e3
Merge branch master into cheme/new-trie2
MattHalpinParity 2640630
Merge branch master into cheme/new-trie2
MattHalpinParity 5413fe0
Fix prdoc
MattHalpinParity c496b68
Fix prdoc
MattHalpinParity 73f72cb
Merge branch master into cheme/new-trie2
MattHalpinParity 18632d0
Split prune_blocks
MattHalpinParity 96572df
Fix
MattHalpinParity File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
mix node conf multi and rocks
- Loading branch information
commit 06e32c909e4e5555dcde7783ee7b32e187911451
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -158,6 +158,7 @@ pub fn node_config( | |
| key: Sr25519Keyring, | ||
| boot_nodes: Vec<MultiaddrWithPeerId>, | ||
| is_validator: bool, | ||
| multi_tree: bool, | ||
| ) -> Configuration { | ||
| let base_path = BasePath::new_temp_dir().expect("could not create temporary directory"); | ||
| let root = base_path.path().join(key.to_string()); | ||
|
|
@@ -195,9 +196,11 @@ pub fn node_config( | |
| transaction_pool: Default::default(), | ||
| network: network_config, | ||
| keystore: KeystoreConfig::InMemory, | ||
| //database: DatabaseSource::RocksDb { path: root.join("db"), cache_size: 128 }, // TODO | ||
| // restore rocksdb? | ||
| database: DatabaseSource::ParityDb { path: root.join("db"), multi_tree: true }, | ||
| database: if multi_tree { | ||
| DatabaseSource::ParityDb { path: root.join("db"), multi_tree: true } | ||
| } else { | ||
| DatabaseSource::RocksDb { path: root.join("db"), cache_size: 128 } | ||
| }, | ||
| trie_cache_maximum_size: Some(64 * 1024 * 1024), | ||
| state_pruning: Default::default(), | ||
| blocks_pruning: BlocksPruning::KeepFinalized, | ||
|
|
@@ -276,7 +279,7 @@ pub fn run_collator_node( | |
| boot_nodes: Vec<MultiaddrWithPeerId>, | ||
| collator_pair: CollatorPair, | ||
| ) -> PolkadotTestNode { | ||
| let config = node_config(storage_update_func, tokio_handle, key, boot_nodes, false); | ||
| let config = node_config(storage_update_func, tokio_handle, key, boot_nodes, false, false); | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here consider exposing multitree choice (actually not having paritydb tested here may not have been really good at the first place). |
||
| let multiaddr = config.network.listen_addresses[0].clone(); | ||
| let NewFull { task_manager, client, network, rpc_handlers, overseer_handle, .. } = new_full( | ||
| config, | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same thing choice of multitree is rather random here (alic multi bob not), but would think it is fine here.