Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Commit 57d2e53

Browse files
gpestanaggwpezdavxyyjhmelodybkchr
authored
[NPoS] Implements dynamic number of nominators (#12970)
* Implements dynamic nominations per nominator * Adds SnapshotBounds and ElectionSizeTracker * Changes the ElectionDataProvider interface to receive ElectionBounds as input * Implements get_npos_voters with ElectionBounds * Implements get_npos_targets with ElectionBounds * Adds comments * tests * Truncates nomninations that exceed nominations quota; Old tests passing * Uses DataProviderBounds and ElectionBounds (to continue) * Finishes conversions - tests passing * Refactor staking in babe mocks * Replaces MaxElectableTargets and MaxElectingVoters with ElectionBounds; Adds more tests * Fixes nits; node compiling * bechmarks * removes nomination_quota extrinsic to request the nomination quota * Lazy quota check, ie. at nominate time only * remove non-working test (for now) * tests lazy nominations quota when quota is lower than current number of nominated targets * Adds runtime API and custom RPC call for clients to query the nominations quota for a given balance * removes old rpc * Cosmetic touches * All mocks working * Fixes benchmarking mocks * nits * more tests * renames trait methods * nit * ".git/.scripts/commands/fmt/fmt.sh" * Fix V2 PoV benchmarking (#13485) * Bump default 'additional_trie_layers' to two The default here only works for extremely small runtimes, which have no more than 16 storage prefices. This is changed to a "sane" default of 2, which is save for runtimes with up to 4096 storage prefices (eg StorageValue). Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Update tests and test weights Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix PoV weights Signed-off-by: Oliver Tale-Yazdi <[email protected]> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_balances * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_message_queue * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton * Fix sanity check >0 would also do as a check, but let's try this. Signed-off-by: Oliver Tale-Yazdi <[email protected]> --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: command-bot <> * Move BEEFY code to consensus (#13484) * Move beefy primitives to consensus dir * Move beefy gadget to client consensus folder * Rename beefy crates * chore: move genesis block builder to chain-spec crate. (#13427) * chore: move genesis block builder to block builder crate. * add missing file * chore: move genesis block builder to sc-chain-spec * Update client/chain-spec/src/genesis.rs Co-authored-by: Bastian Köcher <[email protected]> * Update test-utils/runtime/src/genesismap.rs Co-authored-by: Bastian Köcher <[email protected]> * Update test-utils/runtime/client/src/lib.rs * fix warnings * fix warnings --------- Co-authored-by: Bastian Köcher <[email protected]> * Speed up storage iteration from within the runtime (#13479) * Speed up storage iteration from within the runtime * Move the cached iterator into an `Option` * Use `RefCell` in no_std * Simplify the code slightly * Use `Option::replace` * Update doc comment for `next_storage_key_slow` * Make unbounded channels size warning exact (part 1) (#13490) * Replace `futures-channel` with `async-channel` in `out_events` * Apply suggestions from code review Co-authored-by: Koute <[email protected]> * Also print the backtrace of `send()` call * Switch from `backtrace` crate to `std::backtrace` * Remove outdated `backtrace` dependency * Remove `backtrace` from `Cargo.lock` --------- Co-authored-by: Koute <[email protected]> * Removal of Prometheus alerting rules deployment in cloud-infra (#13499) * sp-consensus: remove unused error variants (#13495) * Expose `ChargedAmount` (#13488) * Expose `ChargedAmount` * Fix imports * sc-consensus-beefy: fix metrics: use correct names (#13494) Signed-off-by: acatangiu <[email protected]> * clippy fix * removes NominationsQuotaExceeded event * Update frame/staking/src/lib.rs Co-authored-by: Ross Bulat <[email protected]> * adds back the npos_max_iter * remove duplicate imports added after merge * fmt * Adds comment in public struct; Refactors CountBound and SizeCount to struct * addresses various pr comments * PR comment reviews * Fixes on-chain election bounds and related code * EPM checks the size of the voter list returned by the data provider * cosmetic changes * updates e2e tests mock * Adds more tests for size tracker and refactors code * Adds back only_iterates_max_2_times_max_allowed_len test * Refactor * removes unecessary dependency * empty commit -- restart all stuck CI jobs * restarts ci jobs * Renames ElectionBounds -> Bounds in benchmarking mocks et al * updates mocks * Update frame/election-provider-support/src/lib.rs Co-authored-by: Kian Paimani <[email protected]> * Update frame/staking/src/pallet/impls.rs Co-authored-by: Kian Paimani <[email protected]> * Update frame/election-provider-support/src/lib.rs Co-authored-by: Kian Paimani <[email protected]> * Update frame/staking/src/tests.rs Co-authored-by: Kian Paimani <[email protected]> * more checks in api_nominations_quota in tests * Improves docs * fixes e2e tests * Uses size_hint rather than mem::size_of in size tracker; Refactor size tracker to own module * nits from reviews * Refactors bounds to own module; improves docs * More tests and docs * fixes docs * Fixes benchmarks * Fixes rust docs * fixes bags-list remote-ext-tests * Simplify bound checks in create_snapshot_external * Adds target size check in get_npos_targets * ".git/.scripts/commands/fmt/fmt.sh" * restart ci * rust doc fixes and cosmetic nits * rollback upgrade on parity-scale-codec version (unecessary) * reset cargo lock, no need to update it --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Signed-off-by: acatangiu <[email protected]> Co-authored-by: command-bot <> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Davide Galassi <[email protected]> Co-authored-by: yjh <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Koute <[email protected]> Co-authored-by: Dmitry Markin <[email protected]> Co-authored-by: Anthony Lazam <[email protected]> Co-authored-by: André Silva <[email protected]> Co-authored-by: Piotr Mikołajczyk <[email protected]> Co-authored-by: Adrian Catangiu <[email protected]> Co-authored-by: Ross Bulat <[email protected]> Co-authored-by: Kian Paimani <[email protected]>
1 parent 19eb56a commit 57d2e53

File tree

30 files changed

+1415
-307
lines changed

30 files changed

+1415
-307
lines changed

bin/node/runtime/src/lib.rs

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424

2525
use codec::{Decode, Encode, MaxEncodedLen};
2626
use frame_election_provider_support::{
27+
bounds::{ElectionBounds, ElectionBoundsBuilder},
2728
onchain, BalancingConfig, ElectionDataProvider, SequentialPhragmen, VoteWeight,
2829
};
2930
use frame_support::{
@@ -562,14 +563,16 @@ parameter_types! {
562563
pub HistoryDepth: u32 = 84;
563564
}
564565

566+
/// Upper limit on the number of NPOS nominations.
567+
const MAX_QUOTA_NOMINATIONS: u32 = 16;
568+
565569
pub struct StakingBenchmarkingConfig;
566570
impl pallet_staking::BenchmarkingConfig for StakingBenchmarkingConfig {
567571
type MaxNominators = ConstU32<1000>;
568572
type MaxValidators = ConstU32<1000>;
569573
}
570574

571575
impl pallet_staking::Config for Runtime {
572-
type MaxNominations = MaxNominations;
573576
type Currency = Balances;
574577
type CurrencyBalance = Balance;
575578
type UnixTime = Timestamp;
@@ -594,6 +597,7 @@ impl pallet_staking::Config for Runtime {
594597
type ElectionProvider = ElectionProviderMultiPhase;
595598
type GenesisElectionProvider = onchain::OnChainExecution<OnChainSeqPhragmen>;
596599
type VoterList = VoterList;
600+
type NominationsQuota = pallet_staking::FixedNominationsQuota<MAX_QUOTA_NOMINATIONS>;
597601
// This a placeholder, to be introduced in the next PR as an instance of bags-list
598602
type TargetList = pallet_staking::UseValidatorsMap<Self>;
599603
type MaxUnlockingChunks = ConstU32<32>;
@@ -647,17 +651,20 @@ frame_election_provider_support::generate_solution_type!(
647651
VoterIndex = u32,
648652
TargetIndex = u16,
649653
Accuracy = sp_runtime::PerU16,
650-
MaxVoters = MaxElectingVoters,
654+
MaxVoters = MaxElectingVotersSolution,
651655
>(16)
652656
);
653657

654658
parameter_types! {
659+
// Note: the EPM in this runtime runs the election on-chain. The election bounds must be
660+
// carefully set so that an election round fits in one block.
661+
pub ElectionBoundsMultiPhase: ElectionBounds = ElectionBoundsBuilder::default()
662+
.voters_count(10_000.into()).targets_count(1_500.into()).build();
663+
pub ElectionBoundsOnChain: ElectionBounds = ElectionBoundsBuilder::default()
664+
.voters_count(5_000.into()).targets_count(1_250.into()).build();
665+
655666
pub MaxNominations: u32 = <NposSolution16 as frame_election_provider_support::NposSolution>::LIMIT as u32;
656-
pub MaxElectingVoters: u32 = 40_000;
657-
pub MaxElectableTargets: u16 = 10_000;
658-
// OnChain values are lower.
659-
pub MaxOnChainElectingVoters: u32 = 5000;
660-
pub MaxOnChainElectableTargets: u16 = 1250;
667+
pub MaxElectingVotersSolution: u32 = 40_000;
661668
// The maximum winners that can be elected by the Election pallet which is equivalent to the
662669
// maximum active validators the staking pallet can have.
663670
pub MaxActiveValidators: u32 = 1000;
@@ -712,8 +719,7 @@ impl onchain::Config for OnChainSeqPhragmen {
712719
type DataProvider = <Runtime as pallet_election_provider_multi_phase::Config>::DataProvider;
713720
type WeightInfo = frame_election_provider_support::weights::SubstrateWeight<Runtime>;
714721
type MaxWinners = <Runtime as pallet_election_provider_multi_phase::Config>::MaxWinners;
715-
type VotersBound = MaxOnChainElectingVoters;
716-
type TargetsBound = MaxOnChainElectableTargets;
722+
type Bounds = ElectionBoundsOnChain;
717723
}
718724

719725
impl pallet_election_provider_multi_phase::MinerConfig for Runtime {
@@ -761,9 +767,8 @@ impl pallet_election_provider_multi_phase::Config for Runtime {
761767
type GovernanceFallback = onchain::OnChainExecution<OnChainSeqPhragmen>;
762768
type Solver = SequentialPhragmen<AccountId, SolutionAccuracyOf<Self>, OffchainRandomBalancing>;
763769
type ForceOrigin = EnsureRootOrHalfCouncil;
764-
type MaxElectableTargets = MaxElectableTargets;
765770
type MaxWinners = MaxActiveValidators;
766-
type MaxElectingVoters = MaxElectingVoters;
771+
type ElectionBounds = ElectionBoundsMultiPhase;
767772
type BenchmarkingConfig = ElectionProviderBenchmarkConfig;
768773
type WeightInfo = pallet_election_provider_multi_phase::weights::SubstrateWeight<Self>;
769774
}

frame/babe/src/mock.rs

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,16 @@
1919
2020
use crate::{self as pallet_babe, Config, CurrentSlot};
2121
use codec::Encode;
22-
use frame_election_provider_support::{onchain, SequentialPhragmen};
22+
use frame_election_provider_support::{
23+
bounds::{ElectionBounds, ElectionBoundsBuilder},
24+
onchain, SequentialPhragmen,
25+
};
2326
use frame_support::{
2427
parameter_types,
2528
traits::{ConstU128, ConstU32, ConstU64, KeyOwnerProofSystem, OnInitialize},
2629
};
2730
use pallet_session::historical as pallet_session_historical;
31+
use pallet_staking::FixedNominationsQuota;
2832
use sp_consensus_babe::{AuthorityId, AuthorityPair, Randomness, Slot, VrfSignature};
2933
use sp_core::{
3034
crypto::{KeyTypeId, Pair, VrfSecret},
@@ -161,6 +165,7 @@ parameter_types! {
161165
pub const SlashDeferDuration: EraIndex = 0;
162166
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
163167
pub const OffendingValidatorsThreshold: Perbill = Perbill::from_percent(16);
168+
pub static ElectionsBounds: ElectionBounds = ElectionBoundsBuilder::default().build();
164169
}
165170

166171
pub struct OnChainSeqPhragmen;
@@ -170,12 +175,10 @@ impl onchain::Config for OnChainSeqPhragmen {
170175
type DataProvider = Staking;
171176
type WeightInfo = ();
172177
type MaxWinners = ConstU32<100>;
173-
type VotersBound = ConstU32<{ u32::MAX }>;
174-
type TargetsBound = ConstU32<{ u32::MAX }>;
178+
type Bounds = ElectionsBounds;
175179
}
176180

177181
impl pallet_staking::Config for Test {
178-
type MaxNominations = ConstU32<16>;
179182
type RewardRemainder = ();
180183
type CurrencyToVote = ();
181184
type RuntimeEvent = RuntimeEvent;
@@ -197,6 +200,7 @@ impl pallet_staking::Config for Test {
197200
type GenesisElectionProvider = Self::ElectionProvider;
198201
type VoterList = pallet_staking::UseNominatorsAndValidatorsMap<Self>;
199202
type TargetList = pallet_staking::UseValidatorsMap<Self>;
203+
type NominationsQuota = FixedNominationsQuota<16>;
200204
type MaxUnlockingChunks = ConstU32<32>;
201205
type HistoryDepth = ConstU32<84>;
202206
type EventListeners = ();

frame/bags-list/remote-tests/src/snapshot.rs

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,10 @@
1616

1717
//! Test to execute the snapshot using the voter bag.
1818
19-
use frame_election_provider_support::SortedListProvider;
19+
use frame_election_provider_support::{
20+
bounds::{CountBound, DataProviderBounds},
21+
SortedListProvider,
22+
};
2023
use frame_support::traits::PalletInfoAccess;
2124
use remote_externalities::{Builder, Mode, OnlineConfig};
2225
use sp_runtime::{traits::Block as BlockT, DeserializeOwned};
@@ -62,8 +65,13 @@ where
6265
<Runtime as pallet_staking::Config>::VoterList::count(),
6366
);
6467

68+
let bounds = match voter_limit {
69+
None => DataProviderBounds::default(),
70+
Some(v) => DataProviderBounds { count: Some(CountBound(v as u32)), size: None },
71+
};
72+
6573
let voters =
66-
<pallet_staking::Pallet<Runtime> as ElectionDataProvider>::electing_voters(voter_limit)
74+
<pallet_staking::Pallet<Runtime> as ElectionDataProvider>::electing_voters(bounds)
6775
.unwrap();
6876

6977
let mut voters_nominator_only = voters

frame/beefy/src/mock.rs

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,10 @@
1717

1818
use std::vec;
1919

20-
use frame_election_provider_support::{onchain, SequentialPhragmen};
20+
use frame_election_provider_support::{
21+
bounds::{ElectionBounds, ElectionBoundsBuilder},
22+
onchain, SequentialPhragmen,
23+
};
2124
use frame_support::{
2225
construct_runtime, parameter_types,
2326
sp_io::TestExternalities,
@@ -184,6 +187,7 @@ parameter_types! {
184187
pub const BondingDuration: EraIndex = 3;
185188
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
186189
pub const OffendingValidatorsThreshold: Perbill = Perbill::from_percent(17);
190+
pub static ElectionsBoundsOnChain: ElectionBounds = ElectionBoundsBuilder::default().build();
187191
}
188192

189193
pub struct OnChainSeqPhragmen;
@@ -193,12 +197,10 @@ impl onchain::Config for OnChainSeqPhragmen {
193197
type DataProvider = Staking;
194198
type WeightInfo = ();
195199
type MaxWinners = ConstU32<100>;
196-
type VotersBound = ConstU32<{ u32::MAX }>;
197-
type TargetsBound = ConstU32<{ u32::MAX }>;
200+
type Bounds = ElectionsBoundsOnChain;
198201
}
199202

200203
impl pallet_staking::Config for Test {
201-
type MaxNominations = ConstU32<16>;
202204
type RewardRemainder = ();
203205
type CurrencyToVote = ();
204206
type RuntimeEvent = RuntimeEvent;
@@ -220,6 +222,7 @@ impl pallet_staking::Config for Test {
220222
type GenesisElectionProvider = Self::ElectionProvider;
221223
type VoterList = pallet_staking::UseNominatorsAndValidatorsMap<Self>;
222224
type TargetList = pallet_staking::UseValidatorsMap<Self>;
225+
type NominationsQuota = pallet_staking::FixedNominationsQuota<16>;
223226
type MaxUnlockingChunks = ConstU32<32>;
224227
type HistoryDepth = ConstU32<84>;
225228
type EventListeners = ();

frame/election-provider-multi-phase/src/benchmarking.rs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
use super::*;
2121
use crate::{unsigned::IndexAssignmentOf, Pallet as MultiPhase};
2222
use frame_benchmarking::account;
23+
use frame_election_provider_support::bounds::DataProviderBounds;
2324
use frame_support::{
2425
assert_ok,
2526
traits::{Hooks, TryCollect},
@@ -270,8 +271,9 @@ frame_benchmarking::benchmarks! {
270271

271272
// we don't directly need the data-provider to be populated, but it is just easy to use it.
272273
set_up_data_provider::<T>(v, t);
273-
let targets = T::DataProvider::electable_targets(None)?;
274-
let voters = T::DataProvider::electing_voters(None)?;
274+
// default bounds are unbounded.
275+
let targets = T::DataProvider::electable_targets(DataProviderBounds::default())?;
276+
let voters = T::DataProvider::electing_voters(DataProviderBounds::default())?;
275277
let desired_targets = T::DataProvider::desired_targets()?;
276278
assert!(<MultiPhase<T>>::snapshot().is_none());
277279
}: {

0 commit comments

Comments
 (0)