Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

test-utils: programatically spawn dev nodes #14704

Merged
paritytech-processbot merged 17 commits into from
Aug 8, 2023
Merged

test-utils: programatically spawn dev nodes #14704

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1b5f0cb
allow spinning up dev node in background without binary
liamaharon Aug 3, 2023
fe8dd35
improve comments
liamaharon Aug 3, 2023
3ad997f
restore rust-toolchain
liamaharon Aug 7, 2023
40ba547
remove rust-toolchain
liamaharon Aug 7, 2023
d54f2fe
tweak start_node_without_binary api
liamaharon Aug 8, 2023
8c28192
Update test-utils/cli/src/lib.rs
liamaharon Aug 8, 2023
fbd585f
address commends
liamaharon Aug 8, 2023
a1ed4b1
merge
liamaharon Aug 8, 2023
2c05aed
use &str
liamaharon Aug 8, 2023
6d42a7a
update example
liamaharon Aug 8, 2023
a5cb426
update comment
liamaharon Aug 8, 2023
e18677f
update docs
liamaharon Aug 8, 2023
6b36155
Revert "update docs"
liamaharon Aug 8, 2023
838f507
use node-cli instead of node-template
liamaharon Aug 8, 2023
f05764b
fix feature
liamaharon Aug 8, 2023
0266ef7
fix feature
liamaharon Aug 8, 2023
b7363eb
fix features
liamaharon Aug 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update docs
  • Loading branch information
@liamaharon
liamaharon committed Aug 8, 2023
commit e18677f081247dd9672f64ea34d2c651daf401c2
334 changes: 3 additions & 331 deletions utils/frame/try-runtime/cli/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,338 +17,10 @@

//! # Try-runtime
//!
//! Substrate's ultimate testing framework for the power users.
//! Substrate's `try-runtime` subcommand has been migrated to a [standalone
//! CLI](https://github.com/paritytech/try-runtime-cli).
//!
//! > As the name suggests, `try-runtime` is a detailed testing framework that gives you a lot of
//! control over what is being executed in which environment. It is recommended that user's first
//! familiarize themselves with substrate in depth, particularly the execution model. It is critical
//! to deeply understand how the wasm/client/runtime interactions, and the runtime apis work in the
//! substrate runtime, before commencing to working with `try-runtime`.
//!
//! #### Resources
//!
//! Some resources about the above:
//!
//! 1. <https://docs.substrate.io/reference/command-line-tools/try-runtime/>
//! 2. <https://www.crowdcast.io/e/substrate-seminar/41>
//! 3. <https://docs.substrate.io/fundamentals/runtime-development/>
//!
//! ---
//!
//! ## Background Knowledge
//!
//! The basis of all try-runtime commands is the same: connect to a live node, scrape its *state*
//! and put it inside a `TestExternalities`, then call into a *specific runtime-api* using the given
//! state and some *runtime*.
//!
//! Alternatively, the state could come from a snapshot file.
//!
//! All of the variables in the above statement are made *italic*. Let's look at each of them:
//!
//! 1. **State** is the key-value pairs of data that comprise the canonical information that any
//! blockchain is keeping. A state can be full (all key-value pairs), or be partial (only pairs
//! related to some pallets/prefixes). Moreover, some keys are especial and are not related to
//! specific pallets, known as [`well_known_keys`] in substrate. The most important of these is
//! the `:CODE:` key, which contains the code used for execution, when wasm execution is chosen.
//!
//! 2. *A runtime-api* call is a call into a function defined in the runtime, *on top of a given
//! state*. Each subcommand of `try-runtime` utilizes a specific *runtime-api*.
//!
//! 3. Finally, the **runtime** is the actual code that is used to execute the aforementioned
//! runtime-api. Everything in this crate assumes wasm execution, which means the runtime that
//! you use is the one stored onchain, namely under the `:CODE:` key.
//!
//! To recap, a typical try-runtime command does the following:
//!
//! 1. Download the state of a live chain, and write to an `externalities`.
//! 2. Overwrite the `:CODE:` with a given wasm blob
//! 3. Test some functionality via calling a runtime-api.
//!
//! ## Usage
//!
//! To use any of the provided commands, [`SharedParams`] must be provided. The most important of
//! which being [`SharedParams::runtime`], which specifies which runtime to use. Furthermore,
//! [`SharedParams::overwrite_state_version`] can be used to alter the state-version (see
//! <https://forum.polkadot.network/t/state-trie-migration/852> for more info).
//!
//! Then, the specific command has to be specified. See [`Command`] for more information about each
//! command's specific customization flags, and assumptions regarding the runtime being used.
//!
//! Said briefly, this CLI is capable of executing:
//!
//! * [`Command::OnRuntimeUpgrade`]: execute all the `on_runtime_upgrade` hooks.
//! * [`Command::ExecuteBlock`]: re-execute the given block.
//! * [`Command::OffchainWorker`]: re-execute the given block's offchain worker code path.
//! * [`Command::FollowChain`]: continuously execute the blocks of a remote chain on top of a given
//! runtime.
//! * [`Command::CreateSnapshot`]: Create a snapshot file from a remote node.
//!
//! Finally, To make sure there are no errors regarding this, always run any `try-runtime` command
//! with `executor=trace` logging targets, which will specify which runtime is being used per api
//! call. Moreover, `remote-ext`, `try-runtime` and `runtime` logs targets will also be useful.
//!
//! ## Spec name check
//!
//! A common pitfall is that you might be running some test on top of the state of chain `x`, with
//! the runtime of chain `y`. To avoid this all commands do a spec-name check before executing
//! anything by default. This will check the, if any alterations are being made to the `:CODE:`,
//! then the spec names match. The spec versions are warned, but are not mandated to match.
//!
//! > If anything, in most cases, we expect spec-versions to NOT match, because try-runtime is all
//! > about testing unreleased runtimes.
//!
//! ## Note on signature and state-root checks
//!
//! All of the commands calling into `TryRuntime_execute_block` ([`Command::ExecuteBlock`] and
//! [`Command::FollowChain`]) disable both state root and signature checks. This is because in 99%
//! of the cases, the runtime that is being tested is different from the one that is stored in the
//! canonical chain state. This implies:
//!
//! 1. the state root will NEVER match, because `:CODE:` is different between the two.
//! 2. replaying all transactions will fail, because the spec-version is part of the transaction
//! signature.
//!
//! ## Best Practices
//!
//! Try-runtime is all about battle-testing unreleased runtime. The following list of suggestions
//! help developers maximize the testing coverage and make base use of `try-runtime`.
//!
//! #### Adding pre/post hooks
//!
//! One of the gems that come only in the `try-runtime` feature flag is the `pre_upgrade` and
//! `post_upgrade` hooks for `OnRuntimeUpgrade`. This trait is implemented either inside the pallet,
//! or manually in a runtime, to define a migration. In both cases, these functions can be added,
//! given the right flag:
//!
//! ```ignore
//! #[cfg(feature = "try-runtime")]
//! fn pre_upgrade() -> Result<Vec<u8>, TryRuntimeError> {}
//!
//! #[cfg(feature = "try-runtime")]
//! fn post_upgrade(state: Vec<u8>) -> Result<(), TryRuntimeError> {}
//! ```
//!
//! (The pallet macro syntax will support this simply as a part of `#[pallet::hooks]`).
//!
//! These hooks allow you to execute some code, only within the `on-runtime-upgrade` command, before
//! and after the migration. Moreover, `pre_upgrade` can return a `Vec<u8>` that contains arbitrary
//! encoded data (usually some pre-upgrade state) which will be passed to `post_upgrade` after
//! upgrading and used for post checking.
//!
//! ## State Consistency
//!
//! Similarly, each pallet can expose a function in `#[pallet::hooks]` section as follows:
//!
//! ```ignore
//! #[cfg(feature = "try-runtime")]
//! fn try_state(_: BlockNumber) -> Result<(), TryRuntimeError> {}
//! ```
//!
//! which is called on numerous code paths in the try-runtime tool. These checks should ensure that
//! the state of the pallet is consistent and correct. See `frame_support::try_runtime::TryState`
//! for more info.
//!
//! #### Logging
//!
//! It is super helpful to make sure your migration code uses logging (always with a `runtime` log
//! target prefix, e.g. `runtime::balance`) and state exactly at which stage it is, and what it is
//! doing.
//!
//! #### Guarding migrations
//!
//! Always make sure that any migration code is guarded either by `StorageVersion`, or by some
//! custom storage item, so that it is NEVER executed twice, even if the code lives in two
//! consecutive runtimes.
//!
//! ## Examples
//!
//! For the following examples, we assume the existence of the following:
//!
//! 1. a substrate node compiled without `--feature try-runtime`, called `substrate`. This will be
//! the running node that you connect to. then, after some changes to this node, you compile it with
//! `--features try-runtime`. This gives you:
//! 2. a substrate binary that has the try-runtime sub-command enabled.
//! 3. a wasm blob that has try-runtime functionality.
//!
//! ```bash
//! # this is like your running deployed node.
//! cargo build --release && cp target/release/substrate .
//!
//! # this is like your WIP branch.
//! cargo build --release --features try-runtime
//! cp target/release/substrate substrate-try-runtime
//! cp ./target/release/wbuild/kitchensink-runtime/kitchensink_runtime.wasm runtime-try-runtime.wasm
//! ```
//!
//! > The above example is with `substrate`'s `kitchensink-runtime`, but is applicable to any
//! > substrate-based chain that has implemented `try-runtime-cli`.
//!
//! * If you run `try-runtime` subcommand against `substrate` binary listed above, you get the
//! following error.
//!
//! ```bash
//! [substrate] ./substrate try-runtime
//! Error: Input("TryRuntime wasn't enabled when building the node. You can enable it with `--features try-runtime`.")
//! ```
//!
//! * If you run the same against `substrate-try-runtime`, it will work.
//!
//! ```bash
//! [substrate] ./substrate-try-runtime try-runtime
//! Try some command against runtime state
//!
//! Usage: substrate-try-runtime try-runtime [OPTIONS] --runtime <RUNTIME> <COMMAND>
//!
//! Commands:
//! on-runtime-upgrade Execute the migrations of the "local runtime"
//! execute-block Executes the given block against some state
//! offchain-worker Executes *the offchain worker hooks* of a given block against some state
//! follow-chain Follow the given chain's finalized blocks and apply all of its extrinsics
//! create-snapshot Create a new snapshot file
//! help Print this message or the help of the given subcommand(s)
//!
//! Options:
//! --chain <CHAIN_SPEC>
//! Specify the chain specification
//! --dev
//! Specify the development chain
//! -d, --base-path <PATH>
//! Specify custom base path
//! -l, --log <LOG_PATTERN>...
//! Sets a custom logging filter. Syntax is `<target>=<level>`, e.g. -lsync=debug
//! --detailed-log-output
//! Enable detailed log output
//! --disable-log-color
//! Disable log color output
//! --enable-log-reloading
//! Enable feature to dynamically update and reload the log filter
//! --tracing-targets <TARGETS>
//! Sets a custom profiling filter. Syntax is the same as for logging: `<target>=<level>`
//! --tracing-receiver <RECEIVER>
//! Receiver to process tracing messages [default: log] [possible values: log]
//! --runtime <RUNTIME>
//! The runtime to use
//! --wasm-execution <METHOD>
//! Type of wasm execution used [default: compiled] [possible values: interpreted-i-know-what-i-do, compiled]
//! --wasm-instantiation-strategy <STRATEGY>
//! The WASM instantiation method to use [default: pooling-copy-on-write] [possible values: pooling-copy-on-write, recreate-instance-copy-on-write, pooling, recreate-instance, legacy-instance-reuse]
//! --heap-pages <HEAP_PAGES>
//! The number of 64KB pages to allocate for Wasm execution. Defaults to [`sc_service::Configuration.default_heap_pages`]
//! --overwrite-state-version <OVERWRITE_STATE_VERSION>
//! Overwrite the `state_version`
//! -h, --help
//! Print help information (use `--help` for more detail)
//! -V, --version
//! Print version information
//! ```
//!
//! * Run the migrations of a given runtime on top of a live state.
//!
//! ```bash
//! # assuming there's `./substrate --dev --tmp --ws-port 9999` or similar running.
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! live --uri ws://localhost:9999
//! ```
//!
//! * Same as the previous one, but run it at specific block number's state. This means that this
//! block hash's state shall not yet have been pruned in `rpc.polkadot.io`.
//!
//! ```bash
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! live --uri ws://localhost:9999 \
//! # replace with your desired block hash!
//! --at 0xa1b16c1efd889a9f17375ec4dd5c1b4351a2be17fa069564fced10d23b9b3836
//! ```
//!
//! * Executing the same command with the [`Runtime::Existing`] will fail because the existing
//! runtime, stored onchain in `substrate` binary that we compiled earlier does not have
//! `try-runtime` feature!
//!
//! ```bash
//! ./substrate-try-runtime try-runtime --runtime existing -lruntime=debug on-runtime-upgrade live --uri ws://localhost:9999
//! ...
//! Error: Input("given runtime is NOT compiled with try-runtime feature!")
//! ```
//!
//! * Now, let's use a snapshot file. First, we create the snapshot:
//!
//! ```bash
//! ./substrate-try-runtime try-runtime --runtime existing -lruntime=debug create-snapshot --uri ws://localhost:9999
//! 2022-12-13 10:28:17.516 INFO main try-runtime::cli: snapshot path not provided (-s), using '[email protected]'
//! 2022-12-13 10:28:17.516 INFO main remote-ext: since no at is provided, setting it to latest finalized head, 0xe7d0b614dfe89af65b33577aae46a6f958c974bf52f8a5e865a0f4faeb578d22
//! 2022-12-13 10:28:17.516 INFO main remote-ext: since no prefix is filtered, the data for all pallets will be downloaded
//! 2022-12-13 10:28:17.550 INFO main remote-ext: writing snapshot of 1611464 bytes to "[email protected]"
//! 2022-12-13 10:28:17.551 INFO main remote-ext: initialized state externalities with storage root 0x925e4e95de4c08474fb7f976c4472fa9b8a1091619cd7820a793bf796ee6d932 and state_version V1
//! ```
//!
//! > Note that the snapshot contains the `existing` runtime, which does not have the correct
//! > `try-runtime` feature. In the following commands, we still need to overwrite the runtime.
//!
//! Then, we can use it to have the same command as before, `on-runtime-upgrade`
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! snap -s [email protected]
//! ```
//!
//! * Execute the latest finalized block with the given runtime.
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! execute-block live \
//! --uri ws://localhost:9999
//! ```
//!
//! This can still be customized at a given block with `--at`. If you want to use a snapshot, you
//! can still use `--block-ws-uri` to provide a node form which the block data can be fetched.
//!
//! Moreover, this runs the `frame_support::try_runtime::TryState` hooks as well. The hooks to run
//! can be customized with the `--try-state`. For example:
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! execute-block \
//! --try-state System,Staking \
//! live \
//! --uri ws://localhost:9999 \
//! --pallet System Staking
//! ```
//!
//! Will only run the `try-state` of the two given pallets. When running `try-state` against
//! some real chain data it can take a long time for the command to execute since it has to
//! query all the key-value pairs. In scenarios like above where we only want to run the
//! `try-state` for some specific pallets, we can use the `--pallet` option to specify from
//! which pallets we want to query the state. This will greatly decrease the execution time.
//!
//! See [`frame_try_runtime::TryStateSelect`] for more information.
//!
//! * Follow our live chain's blocks using `follow-chain`, whilst running the try-state of 3 pallets
//! in a round robin fashion
//!
//! ```bash
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! follow-chain \
//! --uri ws://localhost:9999 \
//! --try-state rr-3
//! ```
//! It is no longer maintained here and will be removed in the future.

#![cfg(feature = "try-runtime")]

Expand Down