Skip to content

Improve TryDecodeEntireState output #2724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 23, 2023
Merged

Improve TryDecodeEntireState output #2724

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c57775d
improve decode err output
liamaharon Dec 15, 2023
d1de7ac
? syntax
liamaharon Dec 15, 2023
587c57b
clean up
liamaharon Dec 15, 2023
06df60d
Merge branch 'master' into liam-improve-decode-err-output
liamaharon Dec 18, 2023
2840411
fix missing first key
liamaharon Dec 18, 2023
01dc683
Merge branch 'liam-improve-decode-err-output' of github.com:paritytec…
liamaharon Dec 18, 2023
cd673c5
remove hex and add test
liamaharon Dec 18, 2023
048fe64
improve log
liamaharon Dec 20, 2023
7dae7a0
fix
liamaharon Dec 20, 2023
8d3ceba
Merge branch 'master' into liam-improve-decode-err-output
liamaharon Dec 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions substrate/frame/executive/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,9 +409,10 @@ where
) -> Result<(), TryRuntimeError> {
match res {
Ok(bytes) => {
log::debug!(
log::info!(
target: LOG_TARGET,
"decoded the entire state ({bytes} bytes)",
"✅ Entire runtime state decodes without error. {} bytes total.",
bytes
);

Ok(())
Expand Down
50 changes: 39 additions & 11 deletions substrate/frame/support/src/traits/try_runtime/decode_entire_state.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ impl TryDecodeEntireStorage for Tuple {
}

/// A value could not be decoded.
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Clone, PartialEq, Eq)]
pub struct TryDecodeEntireStorageError {
/// The key of the undecodable value.
pub key: Vec<u8>,
Expand All @@ -81,9 +81,22 @@ impl core::fmt::Display for TryDecodeEntireStorageError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(
f,
"Failed to decode storage item `{}::{}`",
"`{}::{}` key `{}` is undecodable",
&sp_std::str::from_utf8(&self.info.pallet_name).unwrap_or("<invalid>"),
&sp_std::str::from_utf8(&self.info.storage_name).unwrap_or("<invalid>"),
array_bytes::bytes2hex("0x", &self.key)
)
}
}

impl core::fmt::Debug for TryDecodeEntireStorageError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(
f,
"key: {} value: {} info: {:?}",
array_bytes::bytes2hex("0x", &self.key),
array_bytes::bytes2hex("0x", self.raw.clone().unwrap_or_default()),
self.info
)
}
}
Expand All @@ -96,37 +109,46 @@ impl core::fmt::Display for TryDecodeEntireStorageError {
fn decode_storage_info<V: Decode>(
info: StorageInfo,
) -> Result<usize, Vec<TryDecodeEntireStorageError>> {
let mut next_key = info.prefix.clone();
let mut decoded = 0;

let decode_key = |key: &[u8]| match sp_io::storage::get(key) {
None => Ok(0),
Some(bytes) => {
let len = bytes.len();
let _ = <V as DecodeAll>::decode_all(&mut bytes.as_ref()).map_err(|_| {
vec![TryDecodeEntireStorageError {
TryDecodeEntireStorageError {
key: key.to_vec(),
raw: Some(bytes.to_vec()),
info: info.clone(),
}]
}
})?;

Ok::<usize, Vec<_>>(len)
Ok::<usize, _>(len)
},
};

decoded += decode_key(&next_key)?;
let mut errors = vec![];
let mut next_key = Some(info.prefix.clone());
Copy link
Member

@bkchr bkchr Dec 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should just use PrefixKeyIterator here. Not sure why we have written this here "manually" :P

Copy link
Contributor Author

@liamaharon liamaharon Dec 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disadvantage of KeyPrefixIterator is that it "skips" keys that error decoding. We want to handle the Result instead.

loop {
match sp_io::storage::next_key(&next_key) {
match next_key {
Some(key) if key.starts_with(&info.prefix) => {
decoded += decode_key(&key)?;
next_key = key;
match decode_key(&key) {
Ok(bytes) => {
decoded += bytes;
},
Err(e) => errors.push(e),
};
next_key = sp_io::storage::next_key(&key);
},
_ => break,
}
}

Ok(decoded)
if errors.is_empty() {
Ok(decoded)
} else {
Err(errors)
}
}

impl<Prefix, Value, QueryKind, OnEmpty> TryDecodeEntireStorage
Expand Down Expand Up @@ -353,6 +375,12 @@ mod tests {
// two bytes, cannot be decoded into u32.
sp_io::storage::set(&Map::hashed_key_for(2), &[0u8, 1]);
assert!(Map::try_decode_entire_state().is_err());
assert_eq!(Map::try_decode_entire_state().unwrap_err().len(), 1);

// multiple errs in the same map are be detected
sp_io::storage::set(&Map::hashed_key_for(3), &[0u8, 1]);
assert!(Map::try_decode_entire_state().is_err());
assert_eq!(Map::try_decode_entire_state().unwrap_err().len(), 2);
})
}

Expand Down