Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
611f481
DigestItem::ChangesTrieSignal
svyatonik Jun 25, 2019
8b65b04
introduce changes_trie::State
svyatonik Jun 26, 2019
0cd05df
introduce config activation block
svyatonik Jun 26, 2019
40ed2c2
ChangesTrieSignal::as_new_configuration
svyatonik Jun 26, 2019
c01ecd6
moved well_known_cache_keys to client
svyatonik Jun 27, 2019
73f1933
extracted DbChangesTrieStorage to separate file
svyatonik Jun 27, 2019
315303a
change meaning of none in blockchain cache
svyatonik Jun 28, 2019
67642d1
changes trie config (FULL) cache draft
svyatonik Jul 1, 2019
a7e240d
eliminating const ChangesTrieConfiguration
svyatonik Jul 1, 2019
5256d03
delay pruning
svyatonik Jul 2, 2019
698e2ff
continue elimination
svyatonik Jul 3, 2019
04a8fa0
do not prune CT config from cache
svyatonik Jul 3, 2019
afda822
removed redundant code
svyatonik Jul 3, 2019
e08183d
fix some TODOs
svyatonik Jul 4, 2019
368563f
Merge branch 'master' into changes_tries_update_configuration
svyatonik Jul 24, 2019
01e3792
introduce ConfigurationRange
svyatonik Jul 29, 2019
c1913fc
use Configuration range in build
svyatonik Jul 29, 2019
fb6ae9c
build skewed digest
svyatonik Jul 29, 2019
e429061
remove debug print
svyatonik Jul 29, 2019
e6acae8
extracted surface iterator
svyatonik Jul 30, 2019
68527fb
key_changes works with skewed digests
svyatonik Jul 30, 2019
5a839e5
fix client build
svyatonik Jul 30, 2019
9729912
add test for NeverPrune
svyatonik Jul 30, 2019
bca439a
fix TODO
svyatonik Jul 30, 2019
383795a
fixed some TODOs
svyatonik Jul 30, 2019
f6993f6
more tests
svyatonik Jul 31, 2019
8b70450
fixing TODOs
svyatonik Aug 1, 2019
1b3b565
fixed compilation
svyatonik Aug 1, 2019
fa7d631
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 1, 2019
1de7f77
update runtime version
svyatonik Aug 1, 2019
ce78f66
git rid of large tuple
svyatonik Aug 1, 2019
82e74d5
too long lines
svyatonik Aug 1, 2019
cfb1cf1
config_activation_block -> zero
svyatonik Aug 1, 2019
94979f8
obsolete TODO
svyatonik Aug 1, 2019
dd5b602
removed unjustified expect
svyatonik Aug 1, 2019
7260719
update TODOs with issue number
svyatonik Aug 1, 2019
c563d30
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 6, 2019
4552586
new CT pruning algorithm
svyatonik Aug 7, 2019
62a5aa8
Merge branch 'master' into changes_tries_update_configuration
svyatonik Aug 13, 2019
27eecae
BlockIdOrHeader isn't really required
svyatonik Aug 14, 2019
a158250
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 8, 2019
ce92672
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 11, 2019
9da96a5
removed debug leftovers + some docs
svyatonik Nov 12, 2019
f8a14f5
more docs
svyatonik Nov 12, 2019
68c4db0
more post-merge fixes
svyatonik Nov 12, 2019
d63d65e
more post-merge fixes
svyatonik Nov 12, 2019
9baac30
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 12, 2019
6da8b23
revertes some unnecessary changes
svyatonik Nov 12, 2019
d302d1e
reverted unnecessary changes
svyatonik Nov 12, 2019
f4a82de
fix compilation + unnecessary changes
svyatonik Nov 12, 2019
801169c
(restart CI)
svyatonik Nov 12, 2019
eae183b
fix cache update when finalizing multiple blocks
svyatonik Nov 12, 2019
6d2b4db
fixed tests
svyatonik Nov 12, 2019
d7abf44
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 25, 2019
6b3d12d
Merge branch 'master' into changes_tries_update_configuration
svyatonik Nov 29, 2019
d89c979
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Dec 11, 2019
ed31869
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Dec 17, 2019
af3c925
collect_extrinsics -> set_collect_extrinsics
svyatonik Jan 5, 2020
935fb7b
restore lost test
svyatonik Jan 5, 2020
9da28c8
do not calculate block number twice
svyatonik Jan 5, 2020
bf7fea2
Update primitives/blockchain/src/error.rs
svyatonik Jan 5, 2020
3d570f1
map_err -> unwrap_or
svyatonik Jan 5, 2020
b898191
document get_at Result
svyatonik Jan 5, 2020
7db5596
delete abandoned file
svyatonik Jan 5, 2020
9481f59
added weight for set_changes_trie_config
svyatonik Jan 5, 2020
05630d8
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 5, 2020
06755b0
prefer_configs -> fail_if_disabled
svyatonik Jan 5, 2020
d0b2de2
Update client/api/src/backend.rs
svyatonik Jan 5, 2020
2f795e6
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 5, 2020
c4c8871
Update client/db/src/changes_tries_storage.rs
svyatonik Jan 6, 2020
51fa091
CommitOperation+merge -> CommitOperations
svyatonik Jan 6, 2020
98d1958
Merge branch 'changes_tries_update_configuration' of https://github.c…
svyatonik Jan 7, 2020
46d4721
fixed test compilation
svyatonik Jan 7, 2020
39c5f64
merged two different CTRange structs
svyatonik Jan 9, 2020
ee16f37
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Jan 9, 2020
25452a5
lost file
svyatonik Jan 9, 2020
54f3d47
Merge remote-tracking branch 'origin/master' into changes_tries_updat…
svyatonik Jan 13, 2020
233afad
uggrade db from v0 to v1 (init CT cache + add column)
svyatonik Jan 14, 2020
0997bd6
Merge branch 'master' into changes_tries_update_configuration
gavofyork Jan 16, 2020
7658040
fix after merge
svyatonik Jan 16, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
merged two different CTRange structs
  • Loading branch information
svyatonik committed Jan 9, 2020
commit 39c5f64576bde149480ba0c3be4a17aff676396a
17 changes: 4 additions & 13 deletions client/api/src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

use std::sync::Arc;
use std::collections::HashMap;
use sp_core::ChangesTrieConfiguration;
use sp_core::ChangesTrieConfigurationRange;
use sp_core::offchain::OffchainStorage;
use sp_runtime::{generic::BlockId, Justification, Storage};
use sp_runtime::traits::{Block as BlockT, NumberFor};
Expand Down Expand Up @@ -311,25 +311,16 @@ pub trait Backend<Block, H>: AuxStore + Send + Sync where
fn get_import_lock(&self) -> &RwLock<()>;
}

/// Changes trie configuration range.
#[derive(Debug)]
pub struct ChangesTrieConfigurationRange<Block: BlockT> {
/// Zero block of this configuration. First trie that uses this configuration is build at the next block.
pub zero: (NumberFor<Block>, Block::Hash),
/// End block where last trie that uses this configuration has been build. None if configuration is active.
pub end: Option<(NumberFor<Block>, Block::Hash)>,
/// Configuration itself. None if changes tries are disabled within this range.
pub config: Option<ChangesTrieConfiguration>,
}

/// Changes trie storage that supports pruning.
pub trait PrunableStateChangesTrieStorage<Block: BlockT, H: Hasher>:
StateChangesTrieStorage<H, NumberFor<Block>>
{
/// Get reference to StateChangesTrieStorage.
fn storage(&self) -> &dyn StateChangesTrieStorage<H, NumberFor<Block>>;
/// Get configuration at given block.
fn configuration_at(&self, at: &BlockId<Block>) -> sp_blockchain::Result<ChangesTrieConfigurationRange<Block>>;
fn configuration_at(&self, at: &BlockId<Block>) -> sp_blockchain::Result<
ChangesTrieConfigurationRange<NumberFor<Block>, Block::Hash>
>;
/// Get end block (inclusive) of oldest pruned max-level (or skewed) digest trie blocks range.
/// It is guaranteed that we have no any changes tries before (and including) this block.
/// It is guaranteed that all existing changes tries after this block are not yet pruned (if created).
Expand Down
4 changes: 2 additions & 2 deletions client/api/src/light.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ use sp_runtime::{
},
generic::BlockId
};
use sp_core::ChangesTrieConfiguration;
use sp_core::ChangesTrieConfigurationRange;
use sp_state_machine::StorageProof;
use sp_blockchain::{
HeaderMetadata, well_known_cache_keys, HeaderBackend, Cache as BlockchainCache,
Expand Down Expand Up @@ -96,7 +96,7 @@ pub struct RemoteReadChildRequest<Header: HeaderT> {
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct RemoteChangesRequest<Header: HeaderT> {
/// All changes trie configurations that are valid within [first_block; last_block].
pub changes_trie_configs: Vec<(Header::Number, Option<Header::Number>, ChangesTrieConfiguration)>,
pub changes_trie_configs: Vec<ChangesTrieConfigurationRange<Header::Number, Header::Hash>>,
/// Query changes from range of blocks, starting (and including) with this hash...
pub first_block: (Header::Number, Header::Hash),
/// ...ending (and including) with this hash. Should come after first_block and
Expand Down
8 changes: 5 additions & 3 deletions client/db/src/changes_tries_storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ use codec::{Decode, Encode};
use parking_lot::RwLock;
use sp_blockchain::{Error as ClientError, Result as ClientResult};
use sp_trie::MemoryDB;
use sc_client_api::backend::{PrunableStateChangesTrieStorage, ChangesTrieConfigurationRange};
use sc_client_api::backend::PrunableStateChangesTrieStorage;
use sp_blockchain::{well_known_cache_keys, Cache as BlockchainCache};
use sp_core::{H256, Blake2Hasher, ChangesTrieConfiguration, convert_hash};
use sp_core::{H256, Blake2Hasher, ChangesTrieConfiguration, ChangesTrieConfigurationRange, convert_hash};
use sp_runtime::traits::{
Block as BlockT, Header as HeaderT, NumberFor, One, Zero, CheckedSub,
};
Expand Down Expand Up @@ -385,7 +385,9 @@ where
self
}

fn configuration_at(&self, at: &BlockId<Block>) -> ClientResult<ChangesTrieConfigurationRange<Block>> {
fn configuration_at(&self, at: &BlockId<Block>) -> ClientResult<
ChangesTrieConfigurationRange<NumberFor<Block>, Block::Hash>
> {
self.cache
.get_at(&well_known_cache_keys::CHANGES_TRIE_CONFIG, at)?
.and_then(|(zero, end, encoded)| Decode::decode(&mut &encoded[..]).ok()
Expand Down
6 changes: 5 additions & 1 deletion client/network/src/protocol/light_dispatch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1092,7 +1092,11 @@ pub mod tests {

let (tx, response) = oneshot::channel();
light_dispatch.add_request(&mut network_interface, RequestData::RemoteChanges(RemoteChangesRequest {
changes_trie_configs: vec![(0, None, sp_core::ChangesTrieConfiguration::new(4, 2))],
changes_trie_configs: vec![sp_core::ChangesTrieConfigurationRange {
zero: (0, Default::default()),
end: None,
config: Some(sp_core::ChangesTrieConfiguration::new(4, 2)),
}],
first_block: (1, Default::default()),
last_block: (100, Default::default()),
max_block: (100, Default::default()),
Expand Down
31 changes: 21 additions & 10 deletions client/src/light/fetcher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -116,14 +116,13 @@ impl<E, H, B: BlockT, S: BlockchainStorage<B>> LightDataChecker<E, H, B, S> {
// and now check the key changes proof + get the changes
let mut result = Vec::new();
let proof_storage = InMemoryChangesTrieStorage::with_proof(remote_proof);
for (config_zero, config_end, config) in &request.changes_trie_configs {
let config_range = ChangesTrieConfigurationRange {
config,
zero: config_zero.clone(),
end: config_end.clone(),
};
for config_range in &request.changes_trie_configs {
let result_range = key_changes_proof_check_with_db::<H, _>(
config_range,
ChangesTrieConfigurationRange {
config: config_range.config.as_ref().ok_or(ClientError::ChangesTriesNotSupported)?,
zero: config_range.zero.0,
end: config_range.end.map(|(n, _)| n),
},
&RootsStorage {
roots: (request.tries_roots.0, &request.tries_roots.2),
prev_roots: &remote_roots,
Expand Down Expand Up @@ -572,7 +571,11 @@ pub mod tests {
let local_roots_range = local_roots.clone()[(begin - 1) as usize..].to_vec();
let config = ChangesTrieConfiguration::new(4, 2);
let request = RemoteChangesRequest::<Header> {
changes_trie_configs: vec![(0, None, config)],
changes_trie_configs: vec![sp_core::ChangesTrieConfigurationRange {
zero: (0, Default::default()),
end: None,
config: Some(config),
}],
first_block: (begin, begin_hash),
last_block: (end, end_hash),
max_block: (max, max_hash),
Expand Down Expand Up @@ -628,7 +631,11 @@ pub mod tests {
// check proof on local client
let config = ChangesTrieConfiguration::new(4, 2);
let request = RemoteChangesRequest::<Header> {
changes_trie_configs: vec![(0, None, config)],
changes_trie_configs: vec![sp_core::ChangesTrieConfigurationRange {
zero: (0, Default::default()),
end: None,
config: Some(config),
}],
first_block: (1, b1),
last_block: (4, b4),
max_block: (4, b4),
Expand Down Expand Up @@ -670,7 +677,11 @@ pub mod tests {
let local_roots_range = local_roots.clone()[(begin - 1) as usize..].to_vec();
let config = ChangesTrieConfiguration::new(4, 2);
let request = RemoteChangesRequest::<Header> {
changes_trie_configs: vec![(0, None, config)],
changes_trie_configs: vec![sp_core::ChangesTrieConfigurationRange {
zero: (0, Default::default()),
end: None,
config: Some(config),
}],
first_block: (begin, begin_hash),
last_block: (end, end_hash),
max_block: (max, max_hash),
Expand Down
11 changes: 11 additions & 0 deletions primitives/core/src/changes_trie.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ pub struct ChangesTrieConfiguration {
pub digest_levels: u32,
}

/// Substrate changes trie configuration range.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ChangesTrieConfigurationRange<Number, Hash> {
/// Zero block of configuration.
pub zero: (Number, Hash),
/// Last block of configuration (if configuration has been deactivated at some point).
pub end: Option<(Number, Hash)>,
/// The configuration itself. None if changes tries were disabled in this range.
pub config: Option<ChangesTrieConfiguration>,
}

impl ChangesTrieConfiguration {
/// Create new configuration given digest interval and levels.
pub fn new(digest_interval: u32, digest_levels: u32) -> Self {
Expand Down
2 changes: 1 addition & 1 deletion primitives/core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ mod tests;

pub use self::hash::{H160, H256, H512, convert_hash};
pub use self::uint::U256;
pub use changes_trie::ChangesTrieConfiguration;
pub use changes_trie::{ChangesTrieConfiguration, ChangesTrieConfigurationRange};
#[cfg(feature = "full_crypto")]
pub use crypto::{DeriveJunction, Pair, Public};

Expand Down