Skip to content

Fix and update benchmarks #1494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
svyatonik merged 6 commits into from
Jul 7, 2022
Merged

Fix and update benchmarks #1494

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6c1babe
decrease parameters range in grandpa benchmarks
svyatonik Jul 6, 2022
2b3721a
fix messages benchmarks
svyatonik Jul 6, 2022
d02605c
update all weights
svyatonik Jul 6, 2022
0379d24
dealing with failed test (WiP)
svyatonik Jul 6, 2022
087f71c
Revert "dealing with failed test (WiP)"
svyatonik Jul 7, 2022
503f8c4
proper tests fix
svyatonik Jul 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "dealing with failed test (WiP)" 
This reverts commit 0379d24.
  • Loading branch information
@svyatonik
svyatonik committed Jul 7, 2022
commit 087f71c5d3c3d7f897246f025a0d093963a567f4
16 changes: 8 additions & 8 deletions modules/messages/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ Where:
| Benchmark | Description* |
|---------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| `receive_single_message_proof` | Receives proof of single `EXPECTED_DEFAULT_MESSAGE_LENGTH` message |
| `receive_eight_messages_proof` | Receives proof of eight identical `EXPECTED_DEFAULT_MESSAGE_LENGTH` messages |
| `receive_two_messages_proof` | Receives proof of two identical `EXPECTED_DEFAULT_MESSAGE_LENGTH` messages |
| `receive_single_message_proof_with_outbound_lane_state` | Receives proof of single `EXPECTED_DEFAULT_MESSAGE_LENGTH` message and proof of outbound lane state at the source chain |
| `receive_single_message_proof_1_kb` | Receives proof of single message. The proof has size of approximately 1KB** |
| `receive_single_message_proof_16_kb` | Receives proof of single message. The proof has size of approximately 16KB** |
Expand All @@ -311,9 +311,9 @@ Where:

| Component | How it is computed? | Description |
|-------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `BaseWeight` | `2*receive_single_message_proof - receive_eight_messages_proof` | Weight of receiving and parsing minimal proof |
| `BaseWeight` | `2*receive_single_message_proof - receive_two_messages_proof` | Weight of receiving and parsing minimal proof |
| `OutboundStateDeliveryWeight` | `receive_single_message_proof_with_outbound_lane_state - receive_single_message_proof` | Additional weight when proof includes outbound lane state |
| `MessageDeliveryWeight` | `receive_eight_messages_proof - receive_single_message_proof` | Weight of of parsing and dispatching (without actual dispatch cost) of every message |
| `MessageDeliveryWeight` | `receive_two_messages_proof - receive_single_message_proof` | Weight of of parsing and dispatching (without actual dispatch cost) of every message |
| `MessagesCount` | | Provided by relayer |
| `MessagesDispatchWeight` | | Provided by relayer |
| `ActualProofSize` | | Provided by relayer |
Expand Down Expand Up @@ -377,8 +377,8 @@ benchmark (that assumes that the fee is paid during dispatch) and the `receive_s
| Benchmark | Description |
|-------------------------------------------------------------|------------------------------------------------------------------------------------------|
| `receive_delivery_proof_for_single_message` | Receives proof of single message delivery |
| `receive_delivery_proof_for_eight_messages_by_single_relayer` | Receives proof of two messages delivery. Both messages are delivered by the same relayer |
| `receive_delivery_proof_for_eight_messages_by_two_relayers` | Receives proof of two messages delivery. Messages are delivered by different relayers |
| `receive_delivery_proof_for_two_messages_by_single_relayer` | Receives proof of two messages delivery. Both messages are delivered by the same relayer |
| `receive_delivery_proof_for_two_messages_by_two_relayers` | Receives proof of two messages delivery. Messages are delivered by different relayers |

#### Weight formula

Expand All @@ -394,10 +394,10 @@ Where:

| Component | How it is computed? | Description |
|---------------------------|-----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `BaseWeight` | `2*receive_delivery_proof_for_single_message - receive_delivery_proof_for_eight_messages_by_single_relayer` | Weight of receiving and parsing minimal delivery proof |
| `MessageDeliveryWeight` | `receive_delivery_proof_for_eight_messages_by_single_relayer - receive_delivery_proof_for_single_message` | Weight of confirming every additional message |
| `BaseWeight` | `2*receive_delivery_proof_for_single_message - receive_delivery_proof_for_two_messages_by_single_relayer` | Weight of receiving and parsing minimal delivery proof |
| `MessageDeliveryWeight` | `receive_delivery_proof_for_two_messages_by_single_relayer - receive_delivery_proof_for_single_message` | Weight of confirming every additional message |
| `MessagesCount` | | Provided by relayer |
| `RelayerRewardWeight` | `receive_delivery_proof_for_eight_messages_by_two_relayers - receive_delivery_proof_for_eight_messages_by_single_relayer` | Weight of rewarding every additional relayer |
| `RelayerRewardWeight` | `receive_delivery_proof_for_two_messages_by_two_relayers - receive_delivery_proof_for_two_messages_by_single_relayer` | Weight of rewarding every additional relayer |
| `RelayersCount` | | Provided by relayer |
| `ActualProofSize` | | Provided by relayer |
| `ExpectedProofSize` | `EXTRA_STORAGE_PROOF_SIZE` | Size of proof that we are expecting |
Expand Down
77 changes: 24 additions & 53 deletions modules/messages/src/benchmarking.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,18 +307,18 @@ benchmarks_instance_pallet! {
assert!(T::is_message_dispatched(21));
}

// Benchmark `receive_messages_proof` extrinsic with 8 minimal-weight messages and following conditions:
// Benchmark `receive_messages_proof` extrinsic with two minimal-weight messages and following conditions:
// * proof does not include outbound lane state proof;
// * inbound lane already has state, so it needs to be read and decoded;
// * message is successfully dispatched;
// * message requires all heavy checks done by dispatcher;
// * message dispatch fee is paid at target (this) chain.
//
// The weight of single message delivery could be approximated as
// `(weight(receive_eight_messages_proof) - weight(receive_single_message_proof))/7`.
// `weight(receive_two_messages_proof) - weight(receive_single_message_proof)`.
// This won't be super-accurate if message has non-zero dispatch weight, but estimation should
// be close enough to real weight.
receive_eight_messages_proof {
receive_two_messages_proof {
let relayer_id_on_source = T::bridged_relayer_id();
let relayer_id_on_target = account("relayer", 0, SEED);
T::endow_account(&relayer_id_on_target);
Expand All @@ -328,20 +328,18 @@ benchmarks_instance_pallet! {

let (proof, dispatch_weight) = T::prepare_message_proof(MessageProofParams {
lane: T::bench_lane_id(),
message_nonces: 21..=28,
message_nonces: 21..=22,
outbound_lane_data: None,
size: StorageProofSize::Minimal(EXPECTED_DEFAULT_MESSAGE_LENGTH),
dispatch_fee_payment: DispatchFeePayment::AtTargetChain,
});
}: receive_messages_proof(RawOrigin::Signed(relayer_id_on_target), relayer_id_on_source, proof, 8, dispatch_weight)
}: receive_messages_proof(RawOrigin::Signed(relayer_id_on_target), relayer_id_on_source, proof, 2, dispatch_weight)
verify {
assert_eq!(
crate::InboundLanes::<T, I>::get(&T::bench_lane_id()).last_delivered_nonce(),
28,
22,
);
for nonce in 21..=28 {
assert!(T::is_message_dispatched(nonce));
}
assert!(T::is_message_dispatched(22));
}

// Benchmark `receive_messages_proof` extrinsic with single minimal-weight message and following conditions:
Expand Down Expand Up @@ -521,13 +519,13 @@ benchmarks_instance_pallet! {
}

// Benchmark `receive_messages_delivery_proof` extrinsic with following conditions:
// * single relayer is rewarded for relaying eight messages;
// * single relayer is rewarded for relaying two messages;
// * relayer account does not exist (in practice it needs to exist in production environment).
//
// Additional weight for paying single-message reward to the same relayer could be computed
// as `(weight(receive_delivery_proof_for_eight_messages_by_single_relayer)
// - weight(receive_delivery_proof_for_single_message))/7`.
receive_delivery_proof_for_eight_messages_by_single_relayer {
// as `weight(receive_delivery_proof_for_two_messages_by_single_relayer)
// - weight(receive_delivery_proof_for_single_message)`.
receive_delivery_proof_for_two_messages_by_single_relayer {
let relayers_fund_id = crate::relayer_fund_account_id::<T::AccountId, T::AccountIdConverter>();
let relayer_id: T::AccountId = account("relayer", 0, SEED);
let relayer_balance = T::account_balance(&relayer_id);
Expand All @@ -536,27 +534,15 @@ benchmarks_instance_pallet! {
// send message that we're going to confirm
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();

let relayers_state = UnrewardedRelayersState {
unrewarded_relayer_entries: 1,
messages_in_oldest_entry: 8,
total_messages: 8,
last_delivered_nonce: 8,
messages_in_oldest_entry: 2,
total_messages: 2,
last_delivered_nonce: 2,
};
let mut delivered_messages = DeliveredMessages::new(1, true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
delivered_messages.note_dispatched_message(true);
let proof = T::prepare_message_delivery_proof(MessageDeliveryProofParams {
lane: T::bench_lane_id(),
inbound_lane_data: InboundLaneData {
Expand All @@ -570,17 +556,17 @@ benchmarks_instance_pallet! {
});
}: receive_messages_delivery_proof(RawOrigin::Signed(relayer_id.clone()), proof, relayers_state)
verify {
assert_eq!(OutboundLanes::<T, I>::get(T::bench_lane_id()).latest_received_nonce, 8);
assert_eq!(OutboundLanes::<T, I>::get(T::bench_lane_id()).latest_received_nonce, 2);
}

// Benchmark `receive_messages_delivery_proof` extrinsic with following conditions:
// * two relayers are rewarded for relaying single message each;
// * relayer account does not exist (in practice it needs to exist in production environment).
//
// Additional weight for paying reward to the next relayer could be computed
// as `(weight(receive_delivery_proof_for_eight_messages_by_two_relayers)
// - weight(receive_delivery_proof_for_eight_messages_by_single_relayer)) / 7`.
receive_delivery_proof_for_eight_messages_by_two_relayers {
// as `weight(receive_delivery_proof_for_two_messages_by_two_relayers)
// - weight(receive_delivery_proof_for_two_messages_by_single_relayer)`.
receive_delivery_proof_for_two_messages_by_two_relayers {
let relayers_fund_id = crate::relayer_fund_account_id::<T::AccountId, T::AccountIdConverter>();
let relayer1_id: T::AccountId = account("relayer1", 1, SEED);
let relayer1_balance = T::account_balance(&relayer1_id);
Expand All @@ -591,39 +577,24 @@ benchmarks_instance_pallet! {
// send message that we're going to confirm
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();
send_regular_message::<T, I>();

let mut messages1 = DeliveredMessages::new(1, true);
messages1.note_dispatched_message(true);
messages1.note_dispatched_message(true);
messages1.note_dispatched_message(true);
let mut messages2 = DeliveredMessages::new(5, true);
messages2.note_dispatched_message(true);
messages2.note_dispatched_message(true);
messages2.note_dispatched_message(true);

let relayers_state = UnrewardedRelayersState {
unrewarded_relayer_entries: 2,
messages_in_oldest_entry: 4,
total_messages: 8,
last_delivered_nonce: 8,
messages_in_oldest_entry: 1,
total_messages: 2,
last_delivered_nonce: 2,
};
let proof = T::prepare_message_delivery_proof(MessageDeliveryProofParams {
lane: T::bench_lane_id(),
inbound_lane_data: InboundLaneData {
relayers: vec![
UnrewardedRelayer {
relayer: relayer1_id.clone(),
messages: messages1,
messages: DeliveredMessages::new(1, true),
},
UnrewardedRelayer {
relayer: relayer2_id.clone(),
messages: messages2,
messages: DeliveredMessages::new(2, true),
},
].into_iter().collect(),
last_confirmed_nonce: 0,
Expand All @@ -632,7 +603,7 @@ benchmarks_instance_pallet! {
});
}: receive_messages_delivery_proof(RawOrigin::Signed(relayer1_id.clone()), proof, relayers_state)
verify {
assert_eq!(OutboundLanes::<T, I>::get(T::bench_lane_id()).latest_received_nonce, 8);
assert_eq!(OutboundLanes::<T, I>::get(T::bench_lane_id()).latest_received_nonce, 2);
}
}

Expand Down
Loading