Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
611f481
DigestItem::ChangesTrieSignal
svyatonik Jun 25, 2019
8b65b04
introduce changes_trie::State
svyatonik Jun 26, 2019
0cd05df
introduce config activation block
svyatonik Jun 26, 2019
40ed2c2
ChangesTrieSignal::as_new_configuration
svyatonik Jun 26, 2019
c01ecd6
moved well_known_cache_keys to client
svyatonik Jun 27, 2019
73f1933
extracted DbChangesTrieStorage to separate file
svyatonik Jun 27, 2019
315303a
change meaning of none in blockchain cache
svyatonik Jun 28, 2019
67642d1
changes trie config (FULL) cache draft
svyatonik Jul 1, 2019
a7e240d
eliminating const ChangesTrieConfiguration
svyatonik Jul 1, 2019
5256d03
delay pruning
svyatonik Jul 2, 2019
698e2ff
continue elimination
svyatonik Jul 3, 2019
04a8fa0
do not prune CT config from cache
svyatonik Jul 3, 2019
afda822
removed redundant code
svyatonik Jul 3, 2019
e08183d
fix some TODOs
svyatonik Jul 4, 2019
368563f
Merge branch 'master' into changes_tries_update_configuration
svyatonik Jul 24, 2019
01e3792
introduce ConfigurationRange
svyatonik Jul 29, 2019
c1913fc
use Configuration range in build
svyatonik Jul 29, 2019
fb6ae9c
build skewed digest
svyatonik Jul 29, 2019
e429061
remove debug print
svyatonik Jul 29, 2019
e6acae8
extracted surface iterator
svyatonik Jul 30, 2019
68527fb
key_changes works with skewed digests
svyatonik Jul 30, 2019
5a839e5
fix client build
svyatonik Jul 30, 2019
9729912
add test for NeverPrune
svyatonik Jul 30, 2019
bca439a
fix TODO
svyatonik Jul 30, 2019
383795a
fixed some TODOs
svyatonik Jul 30, 2019
f6993f6
more tests
svyatonik Jul 31, 2019
8b70450
fixing TODOs
svyatonik Aug 1, 2019
1b3b565
fixed compilation
svyatonik Aug 1, 2019
fa7d631
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 1, 2019
1de7f77
update runtime version
svyatonik Aug 1, 2019
ce78f66
git rid of large tuple
svyatonik Aug 1, 2019
82e74d5
too long lines
svyatonik Aug 1, 2019
cfb1cf1
config_activation_block -> zero
svyatonik Aug 1, 2019
94979f8
obsolete TODO
svyatonik Aug 1, 2019
dd5b602
removed unjustified expect
svyatonik Aug 1, 2019
7260719
update TODOs with issue number
svyatonik Aug 1, 2019
c563d30
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 6, 2019
4552586
new CT pruning algorithm
svyatonik Aug 7, 2019
62a5aa8
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 13, 2019
27eecae
BlockIdOrHeader isn't really required
svyatonik Aug 14, 2019
a158250
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 8, 2019
ce92672
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 11, 2019
9da96a5
removed debug leftovers + some docs
svyatonik Nov 12, 2019
f8a14f5
more docs
svyatonik Nov 12, 2019
68c4db0
more post-merge fixes
svyatonik Nov 12, 2019
d63d65e
more post-merge fixes
svyatonik Nov 12, 2019
9baac30
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 12, 2019
6da8b23
revertes some unnecessary changes
svyatonik Nov 12, 2019
d302d1e
reverted unnecessary changes
svyatonik Nov 12, 2019
f4a82de
fix compilation + unnecessary changes
svyatonik Nov 12, 2019
801169c
(restart CI)
svyatonik Nov 12, 2019
eae183b
fix cache update when finalizing multiple blocks
svyatonik Nov 12, 2019
6d2b4db
fixed tests
svyatonik Nov 12, 2019
d7abf44
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 25, 2019
6b3d12d
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 29, 2019
d89c979
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Dec 11, 2019
ed31869
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Dec 17, 2019
af3c925
collect_extrinsics -> set_collect_extrinsics
svyatonik Jan 5, 2020
935fb7b
restore lost test
svyatonik Jan 5, 2020
9da28c8
do not calculate block number twice
svyatonik Jan 5, 2020
bf7fea2
Update primitives/blockchain/src/error.rs
svyatonik Jan 5, 2020
3d570f1
map_err -> unwrap_or
svyatonik Jan 5, 2020
b898191
document get_at Result
svyatonik Jan 5, 2020
7db5596
delete abandoned file
svyatonik Jan 5, 2020
9481f59
added weight for set_changes_trie_config
svyatonik Jan 5, 2020
05630d8
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 5, 2020
06755b0
prefer_configs -> fail_if_disabled
svyatonik Jan 5, 2020
d0b2de2
Update client/api/src/backend.rs
svyatonik Jan 5, 2020
2f795e6
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 5, 2020
c4c8871
Update client/db/src/changes_tries_storage.rs
svyatonik Jan 6, 2020
51fa091
CommitOperation+merge -> CommitOperations
svyatonik Jan 6, 2020
98d1958
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 7, 2020
46d4721
fixed test compilation
svyatonik Jan 7, 2020
39c5f64
merged two different CTRange structs
svyatonik Jan 9, 2020
ee16f37
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Jan 9, 2020
25452a5
lost file
svyatonik Jan 9, 2020
54f3d47
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Jan 13, 2020
233afad
uggrade db from v0 to v1 (init CT cache + add column)
svyatonik Jan 14, 2020
0997bd6
Merge branch 'master' into changes_tries_update_configuration
gavofyork Jan 16, 2020
7658040
fix after merge
svyatonik Jan 16, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' into changes_tries_update_configuration
  • Loading branch information
svyatonik committed Aug 6, 2019
commit c563d30c9b0ef0625a30cfa784f3503916870dbc
46 changes: 24 additions & 22 deletions core/client/db/src/changes_tries_storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

use std::collections::HashMap;
use std::sync::Arc;
use hash_db::Prefix;
use kvdb::{KeyValueDB, DBTransaction};
use parity_codec::Encode;
use parking_lot::{RwLock, RwLockWriteGuard};
Expand Down Expand Up @@ -288,14 +289,15 @@ where
self
}

fn get(&self, key: &H256, _prefix: &[u8]) -> Result<Option<DBValue>, String> {
fn get(&self, key: &H256, _prefix: Prefix) -> Result<Option<DBValue>, String> {
self.db.get(self.changes_tries_column, &key[..])
.map_err(|err| format!("{}", err))
}
}

#[cfg(test)]
mod tests {
use hash_db::EMPTY_PREFIX;
use client::backend::{Backend as ClientBackend, NewBlockState, BlockImportOperation};
use client::blockchain::HeaderBackend as BlockchainHeaderBackend;
use sr_primitives::testing::Header;
Expand All @@ -320,7 +322,7 @@ mod tests {

let storage = backend.changes_tries_storage.storage();
for (key, (val, _)) in changes_trie_update.drain() {
assert_eq!(storage.get(&key, &[]), Ok(Some(val)));
assert_eq!(storage.get(&key, EMPTY_PREFIX), Ok(Some(val)));
}
};

Expand Down Expand Up @@ -463,34 +465,34 @@ mod tests {
let mut tx = DBTransaction::new();
backend.changes_tries_storage.prune(&mut tx, blocks[0], Default::default(), 12).unwrap();
backend.storage.db.write(tx).unwrap();
assert!(backend.changes_tries_storage.get(&root1, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root2, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root3, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root4, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root5, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root6, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root7, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root8, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root1, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root2, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root3, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root4, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root5, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root6, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root7, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root8, EMPTY_PREFIX).unwrap().is_some());

// now simulate finalization of block#16, causing prune of tries at #5..#8
let mut tx = DBTransaction::new();
backend.changes_tries_storage.prune(&mut tx, blocks[0], Default::default(), 16).unwrap();
backend.storage.db.write(tx).unwrap();
assert!(backend.changes_tries_storage.get(&root5, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root6, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root7, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root8, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root5, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root6, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root7, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root8, EMPTY_PREFIX).unwrap().is_none());

// now "change" pruning mode to archive && simulate finalization of block#20
// => no changes tries are pruned, because we never prune in archive mode
backend.changes_tries_storage.min_blocks_to_keep = None;
let mut tx = DBTransaction::new();
backend.changes_tries_storage.prune(&mut tx, blocks[0], Default::default(), 20).unwrap();
backend.storage.db.write(tx).unwrap();
assert!(backend.changes_tries_storage.get(&root9, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root10, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root11, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root12, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root9, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root10, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root11, EMPTY_PREFIX).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root12, EMPTY_PREFIX).unwrap().is_some());
}

#[test]
Expand Down Expand Up @@ -547,15 +549,15 @@ mod tests {
let mut tx = DBTransaction::new();
backend.changes_tries_storage.prune(&mut tx, blocks[1], blocks[5], 5).unwrap();
backend.storage.db.write(tx).unwrap();
assert!(backend.changes_tries_storage.get(&root1, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root2, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root1, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root2, EMPTY_PREFIX).unwrap().is_some());

// now simulate finalization of block#6, causing prune of tries at #2
let mut tx = DBTransaction::new();
backend.changes_tries_storage.prune(&mut tx, blocks[1], blocks[6], 6).unwrap();
backend.storage.db.write(tx).unwrap();
assert!(backend.changes_tries_storage.get(&root2, &[]).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root3, &[]).unwrap().is_some());
assert!(backend.changes_tries_storage.get(&root2, EMPTY_PREFIX).unwrap().is_none());
assert!(backend.changes_tries_storage.get(&root3, EMPTY_PREFIX).unwrap().is_some());
}

#[test]
Expand Down
2 changes: 1 addition & 1 deletion core/client/db/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1304,7 +1304,7 @@ where Block: BlockT<Hash=H256> {}

#[cfg(test)]
pub(crate) mod tests {
use hash_db::HashDB;
use hash_db::{HashDB, EMPTY_PREFIX};
use super::*;
use crate::columns;
use client::backend::Backend as BTrait;
Expand Down
2 changes: 1 addition & 1 deletion core/client/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -649,7 +649,7 @@ impl<B, E, Block, RA> Client<B, E, Block, RA> where
self
}

fn get(&self, key: &H256, prefix: &[u8]) -> Result<Option<DBValue>, String> {
fn get(&self, key: &H256, prefix: Prefix) -> Result<Option<DBValue>, String> {
self.storage.get(key, prefix)
}
}
Expand Down
1 change: 1 addition & 0 deletions core/client/src/in_mem.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ use sr_primitives::generic::BlockId;
use sr_primitives::traits::{Block as BlockT, Header as HeaderT, Zero, NumberFor};
use sr_primitives::{Justification, StorageOverlay, ChildrenStorageOverlay};
use state_machine::backend::{Backend as StateBackend, InMemory};

use hash_db::Hasher;
use trie::MemoryDB;

Expand Down
2 changes: 1 addition & 1 deletion core/state-machine/src/changes_trie/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ pub use self::changes_iterator::{
pub use self::prune::{prune, oldest_non_pruned_trie};

use std::convert::TryInto;
use hash_db::Hasher;
use hash_db::{Hasher, Prefix};
use crate::backend::Backend;
use num_traits::{One, Zero};
use parity_codec::{Decode, Encode};
Expand Down
6 changes: 3 additions & 3 deletions core/state-machine/src/changes_trie/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
//! Changes trie storage utilities.

use std::collections::BTreeMap;
use hash_db::{Hasher, Prefix};
use hash_db::{Hasher, Prefix, EMPTY_PREFIX};
use trie::DBValue;
use trie::MemoryDB;
use parking_lot::RwLock;
Expand Down Expand Up @@ -69,7 +69,7 @@ impl<H: Hasher, Number: BlockNumber> InMemoryStorage<H, Number> {

let mut proof_db = MemoryDB::<H>::default();
for item in proof {
proof_db.insert(&[], &item);
proof_db.insert(EMPTY_PREFIX, &item);
}
Self::with_db(proof_db)
}
Expand Down Expand Up @@ -147,7 +147,7 @@ impl<H: Hasher, Number: BlockNumber> Storage<H, Number> for InMemoryStorage<H, N
self
}

fn get(&self, key: &H::Out, prefix: &[u8]) -> Result<Option<DBValue>, String> {
fn get(&self, key: &H::Out, prefix: Prefix) -> Result<Option<DBValue>, String> {
MemoryDB::<H>::get(&self.data.read().mdb, key, prefix)
}
}
Expand Down
4 changes: 4 additions & 0 deletions core/state-machine/src/ext.rs
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,8 @@ mod tests {
let state = Some(ChangesTrieState::new(changes_trie_config(), Zero::zero(), &storage));
let backend = TestBackend::default();
let mut ext = TestExt::new(&mut overlay, &backend, state.as_ref(), None);
let root = hex!("bb0c2ef6e1d36d5490f9766cfcc7dfe2a6ca804504c3bb206053890d6dd02376").into();

assert_eq!(ext.storage_changes_root(Default::default()).unwrap(),
Some(root));
}
Expand All @@ -400,6 +402,8 @@ mod tests {
let state = Some(ChangesTrieState::new(changes_trie_config(), Zero::zero(), &storage));
let backend = TestBackend::default();
let mut ext = TestExt::new(&mut overlay, &backend, state.as_ref(), None);
let root = hex!("96f5aae4690e7302737b6f9b7f8567d5bbb9eac1c315f80101235a92d9ec27f4").into();

assert_eq!(ext.storage_changes_root(Default::default()).unwrap(),
Some(root));
}
Expand Down
4 changes: 2 additions & 2 deletions node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ pub const VERSION: RuntimeVersion = RuntimeVersion {
// and set impl_version to equal spec_version. If only runtime
// implementation changes and behavior does not, then leave spec_version as
// is and increment impl_version.
spec_version: 126,
impl_version: 127,
spec_version: 130,
impl_version: 130,
apis: RUNTIME_API_VERSIONS,
};

Expand Down
You are viewing a condensed version of this merge commit. You can view the full changes here.