Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
f6c4bb2
Straight forward move of trie related only change from child-trie-soft
cheme Mar 26, 2019
878a7ae
merge backend stuff from old branch, add insertion of child trie root in
cheme Mar 26, 2019
a464936
commit before wasm_executor change (we will keep old interface for now:
cheme Mar 26, 2019
4de73d7
tabify
cheme Mar 26, 2019
be0e340
Things compile (account change is unoptimized), not for wasm.
cheme Mar 26, 2019
7f3a282
Make SutrieField non public
cheme Mar 27, 2019
4d208db
Compile no std child trie with subtrie query (super costy: do we trust
cheme Mar 27, 2019
6d7f355
Make the single child trie test pass.
cheme Mar 27, 2019
25bcb4c
Proving synch child trie content (TODO it requires deletion tests).
cheme Mar 28, 2019
9eaef35
restore test
cheme Apr 1, 2019
37d9536
Merge branch 'master' into child-trie-soft-min
cheme Apr 1, 2019
67c03a2
Fix test error (wrong vec alloc).
cheme Apr 2, 2019
99dbb5a
tests for no key collision
cheme Apr 3, 2019
3be1802
Merge branch 'master' into child-trie-soft-min
cheme Apr 3, 2019
fec73d0
Revert `storage_root` using child, create a variant for it: see
cheme Apr 4, 2019
20d168d
indentation and remove comment
cheme Apr 4, 2019
d797bd0
Remove some TODOs, fix compile error
cheme Apr 4, 2019
27096aa
plaindb does not need to be keyspaceddb
cheme Apr 4, 2019
3560acc
Merge branch 'master' into child-trie-soft-min
cheme Apr 5, 2019
c3fc432
bump impl_version
cheme Apr 8, 2019
3e417eb
Merge branch 'master' into child-trie-soft-min
cheme Apr 16, 2019
5af490f
fix and 2299
cheme Apr 16, 2019
5a8dddd
Merge branch 'master' into child-trie-soft-min
cheme Apr 16, 2019
8f04f00
Merge branch 'master' into child-trie-soft-min
cheme Apr 26, 2019
8051179
fix keyspace insert and start proof (missing in higher level test case a
cheme Apr 29, 2019
8eac118
remove todo of test
cheme Apr 30, 2019
a65f9f2
Merge branch 'master' into child-trie-soft-min
cheme Apr 30, 2019
d495ef5
Merge branch 'master' into child-trie-soft-min &
cheme May 14, 2019
197d77a
- refactor api, use noderef of read only query (require and additional
cheme May 15, 2019
2ea3c89
Remove SubTrieNode struct
cheme May 16, 2019
99c45ea
put extension in encoding/decoding properly
cheme May 16, 2019
3553ab2
SubTrie struct renaming (and single encode point)
cheme May 16, 2019
47984fe
remove contract prefix, and make use of a child trie prefix.
cheme May 16, 2019
2bf2d7b
fix for overlay before reverting
cheme May 16, 2019
bc7165c
fix2 for prefix in overlay
cheme May 16, 2019
21c3acf
revert (will probably need two variant, but for rpc pr)
cheme May 16, 2019
98b2fc3
break some lines
cheme May 16, 2019
432cb10
Merge branch 'master' into child-trie-soft-min
cheme May 16, 2019
e266dfe
Add module specific seed to generator (accountid is undefined)
cheme May 16, 2019
7f64652
Update core/primitives/src/subtrie.rs
cheme May 17, 2019
999a26e
Merge branch 'master' into child-trie-soft-min and adapt rpc test
cheme May 21, 2019
7bbd681
Merge branch 'child-trie-soft-min' of github.com:cheme/substrate into…
cheme May 21, 2019
87f03b7
child trie additional prefix is bad design for primitive, removing it.
cheme May 21, 2019
f18e002
safer encoding of version.
cheme May 24, 2019
0eaeca0
Merge branch 'master' into child-trie-soft-min (untested)
cheme May 24, 2019
b57319d
missing import
cheme May 24, 2019
1f848d2
bump impl version
cheme May 24, 2019
403df51
Merge branch 'master' into child-trie-soft-min
cheme May 24, 2019
305b60a
Merge branch 'master' into child-trie-soft-min
cheme May 28, 2019
423cfb1
correct version
cheme May 28, 2019
a0ffa31
fix typing
cheme May 28, 2019
6e3bed7
Merge branch 'master' into child-trie-soft-min
cheme May 29, 2019
7411146
misnamed storage key variables.
cheme May 29, 2019
331be51
fix error messages.
cheme May 29, 2019
bc2935c
remove convenience constructor, remove unneeded pub on field.
cheme May 29, 2019
5a87b6a
Merge branch 'master' into child-trie-soft-min
cheme May 29, 2019
f40400b
minor tweaks before reworking docs
cheme May 30, 2019
9bc1ab7
First pass of documentation improvement.
cheme May 30, 2019
c073b21
do not refer to parent_key when it is ParentTrie:
cheme May 30, 2019
d15ca49
child trie renaming did exceed 100 character line.
cheme May 30, 2019
0736b96
Link github issue for TODOs
cheme May 31, 2019
9e0485d
Merge branch 'master' into child-trie-soft-min
cheme May 31, 2019
586b50e
comment incorrect variable
cheme May 31, 2019
65d7485
Merge branch 'master' into child-trie-soft-min
cheme Jun 6, 2019
95a69b2
Makes child trie instantiation a bisschen more explicit.
cheme Jun 6, 2019
d089693
Merge branch 'master' into child-trie-soft-min
cheme Jun 12, 2019
ec69ae0
rename fetch and init trie function.
cheme Jun 12, 2019
76ea14d
moved_child is a bad idea, just delete in top, if it is reinserted this
cheme Jun 14, 2019
b2050c8
moved map is correct (fix usage of child trie to return option<option<)
cheme Jun 14, 2019
5a0cbe1
Add missing wasm boilerplate.
cheme Jun 14, 2019
6e84810
remove moved_ct from overlay in favor to optional in pending.
cheme Jun 14, 2019
7f5694f
Merge branch 'master' into child-trie-soft-min
cheme Jun 14, 2019
5006d73
Use keyspace for storage_cache.
cheme Jun 17, 2019
b85508e
Comment of unsafety of `set_child` if directly exposed.
cheme Jun 17, 2019
0c14777
Merge branch 'master' into child-trie-soft-min
cheme Jun 17, 2019
10c4f58
use named fields.
cheme Jun 17, 2019
453927b
Merge branch 'master' into child-trie-soft-min
cheme Jun 21, 2019
349f9a5
Merge branch 'master' into child-trie-soft-min
cheme Jun 25, 2019
fc034fb
Refacto of keyspace to use static creation unique info.
cheme Jun 27, 2019
45cfbd6
put child trie content under their own prefix (with common root with
cheme Jun 28, 2019
4f9717e
Revert to storing keyspace in state approach.
cheme Jun 28, 2019
7cb2d84
invalid asumption for `child_delta_trie_root`.
cheme Jun 28, 2019
c821a08
clean unused child storage key relative methods.
cheme Jun 28, 2019
c3ba830
Merge branch 'master' into child-trie-soft-min and
cheme Jun 28, 2019
acf9641
invalid comment
cheme Jun 28, 2019
0050457
indent and remove deprecated comments/struct
cheme Jun 28, 2019
c36b91b
Make top trie write through keyspace db, this is not going fine because
cheme Jul 2, 2019
9d3d9e2
Fix issue: prefixed_key already got the keyspace in prefix.
cheme Jul 2, 2019
3db4da8
Merge branch 'master' into child-trie-soft-min
cheme Jul 2, 2019
4348d70
Address review comment:
cheme Jul 4, 2019
134a4bf
Merge branch 'master' into child-trie-soft-min
cheme Jul 4, 2019
5325621
Restore auth version.
cheme Jul 4, 2019
97118e8
Avoid confusing Option<Option< in result return of overlay.
cheme Jul 4, 2019
0ed7f80
Missing fix from review on testing ext for set child trie (code from
cheme Jul 4, 2019
8a6986a
Use fields for change trie storage.
cheme Jul 4, 2019
7652de9
switch MapTransaction to a struct.
cheme Jul 4, 2019
94c629e
Merge branch 'master' into child-trie-soft-min
cheme Jul 4, 2019
7675740
Apply same encoding for parent root (compatible with implementation of a
cheme Jul 1, 2019
90fba8a
Fix previous cherry pick
cheme Jul 4, 2019
d8c58c6
Addressing nits.
cheme Jul 4, 2019
661ba2e
Avoid unused creation of child trie.
cheme Jul 4, 2019
833e9ff
Create child trie id from additional storage state.
cheme Jul 5, 2019
71bda12
update default encoding.
cheme Jul 5, 2019
08b3062
Handle possible switch to multiple trie layer.
cheme Jul 5, 2019
89f3cd5
Remove a redundancy on keyspaceddb instantiation.
cheme Jul 5, 2019
834f52a
actually remove redundancy correctly
cheme Jul 5, 2019
0445228
comment
cheme Jul 5, 2019
75add99
Merge branch 'child-trie-soft-min-old' into child-trie-soft-min
cheme Jul 5, 2019
e70edab
Merge branch 'master' into child-trie-soft-min and apply review
cheme Jul 9, 2019
b8a0cd4
Merge branch 'master' into child-trie-soft-min
cheme Jul 10, 2019
1514171
Merge branch 'master' into child-trie-soft-min
cheme Jul 10, 2019
a7da811
Fix new test cases.
cheme Jul 10, 2019
b96c523
Merge branch 'master' into child-trie-soft-min
cheme Jul 11, 2019
865672d
Merge branch 'master' into child-trie-soft-min
cheme Jul 12, 2019
b6d7705
bump runtime version
cheme Jul 12, 2019
b45344c
Merge branch 'master' into child-trie-soft-min
cheme Jul 18, 2019
517f95c
Merge branch 'master' into child-trie-soft-min
cheme Jul 31, 2019
79a07de
Merge branch 'master' into child-trie-soft-min
cheme Aug 2, 2019
a027fb0
Some doc enhancement.
cheme Aug 2, 2019
b92655c
ChildRead correctly defined (previous definition was wrong).
cheme Aug 2, 2019
8a875f4
child_trie 'is_updatable_with' to avoid already broken redundancy.
cheme Aug 2, 2019
fcd8bdb
fix system bench error.
cheme Aug 2, 2019
3241fc4
Fix client doc rust code.
cheme Aug 2, 2019
ecac03b
Merge branch 'master' into child-trie-soft-min
cheme Aug 2, 2019
10369b9
Merge branch 'master' into child-trie-soft-min
cheme Aug 7, 2019
15a2967
Merge branch 'master' into child-trie-soft-min
cheme Aug 8, 2019
acc94e4
Merge branch 'master' into child-trie-soft-min
cheme Aug 8, 2019
7d96338
Remove unused method (in favor of assimilate).
cheme Aug 8, 2019
ba7bdcb
Merge branch 'child-trie-soft-min' of github.com:cheme/polkadot into …
cheme Aug 14, 2019
aacea85
Merge branch 'master' into child-trie-soft-min
cheme Aug 14, 2019
6789641
merge fix.
cheme Aug 14, 2019
2b37160
Rename MapTransaction to StorageContent.
cheme Aug 14, 2019
b1183bd
Get subscription stuff from child-deletion branch:
cheme Aug 14, 2019
bc5653c
Merge branch 'master' into child-trie-soft-min
cheme Aug 22, 2019
5ecfd19
update to master.
cheme Aug 22, 2019
baf89c8
Cast explicitelly for readability.
cheme Aug 26, 2019
0f8bff9
bump spec version.
cheme Aug 26, 2019
5a8576b
Merge branch 'master' into child-trie-soft-min
cheme Aug 29, 2019
045ee32
Merge branch 'master' into child-trie-soft-min
cheme Aug 29, 2019
58e6e41
Fix compilation.
cheme Aug 29, 2019
70555c5
Merge branch 'master' into child-trie-soft-min with conflicts.
cheme Sep 5, 2019
130e5e4
build passing, regression on extrinsics for a set_child (see new TODO).
cheme Sep 6, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Some doc enhancement.
  • Loading branch information
cheme committed Aug 2, 2019
commit a027fb0fcdf6247195247023bfcb485d3dd1e47f
35 changes: 18 additions & 17 deletions core/executor/src/wasm_executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -163,17 +163,17 @@ impl_function_executor!(this: FunctionExecutor<'e, E>,
},
ext_child_trie(
storage_key_data: *const u8,
storage_key_len: u32,
k: *mut *mut u8,
kl: *mut u32,
r: *mut *mut u8,
rl: *mut u32,
p: *mut *mut u8,
pl: *mut u32,
e: *mut *mut u8,
el: *mut u32
storage_key_length: u32,
key: *mut *mut u8,
key_length: *mut u32,
root: *mut *mut u8,
root_length: *mut u32,
parent: *mut *mut u8,
parent_length: *mut u32,
extension: *mut *mut u8,
extension_length: *mut u32
) -> u32 => {
let storage_key = this.memory.get(storage_key_data, storage_key_len as usize)
let storage_key = this.memory.get(storage_key_data, storage_key_length as usize)
.map_err(|_| "Invalid attempt to determine storage_key in ext_get_child_trie")?;

Ok(if let Some(ct) = this.ext.child_trie(&storage_key) {
Expand All @@ -195,10 +195,10 @@ impl_function_executor!(this: FunctionExecutor<'e, E>,
Ok(())
};
let fields = ct.to_ptr_vec();
alloc_vec(Some(fields.0), k, kl)?;
alloc_vec(fields.1, r, rl)?;
alloc_vec(Some(fields.2), p, pl)?;
alloc_vec(Some(fields.3), e, el)?;
alloc_vec(Some(fields.0), key, key_length)?;
alloc_vec(fields.1, root, root_length)?;
alloc_vec(Some(fields.2), parent, parent_length)?;
alloc_vec(Some(fields.3), extension, extension_length)?;
1
} else {
0
Expand Down Expand Up @@ -574,8 +574,9 @@ impl_function_executor!(this: FunctionExecutor<'e, E>,
) -> *mut u8 => {
let storage_key = this.memory.get(storage_key_data, storage_key_len as usize)
.map_err(|_| "Invalid attempt to determine storage_key in ext_child_storage_root")?;
let value = this.with_child_trie(&storage_key[..], |this, child_trie|
this.ext.child_storage_root(&child_trie)
let value = this.with_child_trie(
&storage_key[..],
|this, child_trie| this.ext.child_storage_root(&child_trie),
)?;

let offset = this.heap.allocate(value.len() as u32)? as u32;
Expand Down Expand Up @@ -921,7 +922,7 @@ impl_function_executor!(this: FunctionExecutor<'e, E>,
data: *const u8,
data_len: u32,
sig_data_len: *mut u32
) -> *mut u8 => {
) -> *mut u8 => {
let key = offchain::CryptoKey::try_from(key)
.map_err(|_| "Key OOB while ext_sign: wasm")?;
let message = this.memory.get(data, data_len as usize)
Expand Down
2 changes: 1 addition & 1 deletion core/network/src/protocol/on_demand.rs
Original file line number Diff line number Diff line change
Expand Up @@ -980,7 +980,7 @@ pub mod tests {
block: Default::default(),
child_trie: ChildTrieRead {
parent: b"keyspace".to_vec(),
// dummy : this should be queried
// A dummy value: should not be queried.
root: b"root".to_vec(),
},
key: b":key".to_vec(),
Expand Down
54 changes: 32 additions & 22 deletions core/primitives/src/child_trie.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,36 +24,47 @@ use crate::storage::well_known_keys::CHILD_STORAGE_KEY_PREFIX;
pub use impl_serde::serialize as bytes;
use hash_db::Hasher;

/// `KeySpace` type contains a unique identifier to use for accessing
/// child trie node in a underlying key value database.
/// From a `TrieDB` perspective, accessing a child trie content requires
/// both the child trie root, but also the `KeySpace` used to store
/// this child trie.
/// The `KeySpace` of a child trie must be unique for the canonical chain
/// in order to avoid key collision at a key value database level.
/// This id is unique is currently build from a simple counter in state.
/// `KeySpace` type contains a unique child trie identifier.
/// It is used to isolate a child trie in its backing database.
/// This is done by prefixing its key with this value.
///
/// A keyspace byte representation must not be the start of another
/// keyspace byte representation (otherwhise conflict may happen).
/// This guaranty is provided by the fact that keyspace is a scale
/// encoded representation.
///
/// From a `TrieDB` perspective, accessing a child trie content
/// will requires both the child trie root, but also the `KeySpace`.
///
/// The `KeySpace` of a child trie must be unique for the canonical chain.
/// This unicity is currently guaranted by build from a simple counter.
///
/// If a child trie was to be moved between two chains, the underlying
/// key value would be all the keyvaluedb prefixed by this keyspace.
/// Moving those key will still need for every content to change keyspace
/// of the origin chain with a new keyspace of a destination chain.
/// Please notice that usage of keyspace on ALL data makes it possible,
///
/// The same thing is true for a child trie deletion, there is no need to
/// remove all historic of state child trie keypair from a multiple TrieDB
/// perspective, but simply all keyvaluedb content with a key starting with
/// this keyspace.
pub type KeySpace = Vec<u8>;


#[cfg(not(feature = "legacy-trie"))]
/// Keyspace to use for the parent trie key.
pub const NO_CHILD_KEYSPACE: [u8;1] = [0];
#[cfg(feature = "legacy-trie")]
// Keyspace to use for the parent trie key.
const NO_CHILD_KEYSPACE: [u8;0] = [];


/// Generate a new keyspace for a child trie.
pub fn generate_keyspace(child_counter: u128) -> Vec<u8> {
/// Produce a new keyspace from current state counter.
pub fn produce_keyspace(child_counter: u128) -> Vec<u8> {
parity_codec::Encode::encode(&Compact(child_counter))
}

// see FIXME #2741 for removal of this allocation on every operation.
// Simplest would be to put an additional optional field in prefix.
/// Utility function used for merging `KeySpace` data and `prefix` data
/// before calling key value database primitives.
/// Note that it currently does a costly append operation resulting in bigger
/// key length but possibly allowing prefix related operation at lower level.
pub fn keyspace_as_prefix_alloc(ks: Option<&KeySpace>, prefix: &[u8]) -> Vec<u8> {
let ks = ks.map(|ks| ks.as_slice()).unwrap_or(&NO_CHILD_KEYSPACE[..]);
let mut res = rstd::vec![0; ks.len() + prefix.len()];
Expand All @@ -72,10 +83,10 @@ pub fn prefixed_key<H: Hasher>(key: &H::Out, prefix: &[u8]) -> Vec<u8> {
}

/// Parent trie origin. This type contains all information
/// needed to access a parent trie.
/// needed to access a child trie from a root state.
/// Currently only a single depth is supported for child trie,
/// so it only contains the top trie key to the child trie.
/// Internally this contains a full path key (with
/// Internally this contains a full path key (including
/// `well_known_keys::CHILD_STORAGE_KEY_PREFIX`).
pub type ParentTrie = Vec<u8>;

Expand Down Expand Up @@ -200,7 +211,7 @@ impl ChildTrie {
.unwrap_or_else(|| {
let parent = Self::prefix_parent_key(parent);
let ct = ChildTrie {
keyspace: generate_keyspace(child_trie_next_counter()),
keyspace: produce_keyspace(child_trie_next_counter()),
root: Default::default(),
parent,
extension: Default::default(),
Expand Down Expand Up @@ -365,8 +376,7 @@ impl AsRef<ChildTrie> for ChildTrie {

#[test]
fn encode_empty_prefix() {
let empt = generate_keyspace(0);
let empty = produce_keyspace(0);

// this ensure root trie can be move to be a child trie
assert_eq!(&NO_CHILD_KEYSPACE[..], &empt[..]);
assert_eq!(&NO_CHILD_KEYSPACE[..], &empty[..]);
}
3 changes: 0 additions & 3 deletions core/trie/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ hex-literal = "0.2.0"

[features]
default = ["std"]
# use legacy trie implementation
# not compatible with child trie
legacy-trie = []
std = [
"rstd/std",
"codec/std",
Expand Down