Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add rationale for picking batch numbers
  • Loading branch information
athei committed Mar 16, 2023
commit 911d14c8531080dc451b6c091adb51693a8d82ca
9 changes: 8 additions & 1 deletion frame/contracts/src/benchmarking/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,16 @@ use sp_std::prelude::*;
use wasm_instrument::parity_wasm::elements::{BlockType, BrTableData, Instruction, ValueType};

/// How many batches we do per API benchmark.
///
/// This is picked more or less arbitrary. We experimented with different numbers until
/// the results appeared to be stable. Reducing the number would speed up the benchmarks
/// but might make the results less precise.
const API_BENCHMARK_BATCHES: u32 = 1600;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could mention in the comment why these specific values were picked

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking, this is not really batches anymore, is it?

Suggested change
const API_BENCHMARK_BATCHES: u32 = 1600;
const API_BENCHMARK_RUNS: u32 = 1600;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


/// How many batches we do per Instruction benchmark.
/// How many batches we do per instruction benchmark.
///
/// Same rationale as for [`API_BENCHMARK_BATCHES`]. The number is bigger because instruction
/// benchmarks are faster.
const INSTR_BENCHMARK_BATCHES: u32 = 5000;

/// An instantiated and deployed contract.
Expand Down