Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
25bddc9
mockup
kianenigma Sep 11, 2020
7bef52f
Add files
kianenigma Sep 11, 2020
4f3dfa4
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Sep 23, 2020
bc30ce1
More mockup
kianenigma Sep 24, 2020
9e52fbc
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 1, 2020
14ebc95
Lots more changes and tests. Initial design of signed is forseable now.
kianenigma Oct 2, 2020
0587fc9
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 8, 2020
233d4e1
More changes for the signed phase, now going to the cluster-fu*k of g…
kianenigma Oct 9, 2020
9c880f3
Finally decoupled solution from the pallet.. such a PITA.
kianenigma Oct 9, 2020
0cb35eb
Much simplifications and cleanups.
kianenigma Oct 9, 2020
c1ff3ec
A skeleton for the unsigned version.
kianenigma Oct 9, 2020
85a346e
Master.into()
kianenigma Oct 13, 2020
6a2542d
Staking builds and tests compile, some tests fail
kianenigma Oct 13, 2020
c361911
All staking tests work
kianenigma Oct 14, 2020
218db10
Unsigned almost done, primitive crate added.
kianenigma Oct 15, 2020
5e21a79
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 15, 2020
25c1bc3
Use decl_error
kianenigma Oct 15, 2020
d53785c
Add basic benchmark setup, needs more work
kianenigma Oct 16, 2020
525f6e0
Comment.
kianenigma Oct 16, 2020
3026316
Checkpoint.
kianenigma Oct 20, 2020
9841e3b
node-runtime builds.
kianenigma Oct 20, 2020
16c0550
Builds and all good?
kianenigma Dec 3, 2020
f197b6b
Undo some formatting mistakes
kianenigma Dec 3, 2020
5005281
Working on benchmarks'
kianenigma Dec 7, 2020
bd0d331
Fix bench
kianenigma Dec 8, 2020
67dd4d1
Strange merge -- everything builds.
kianenigma Dec 8, 2020
eac4571
Everything compiles and good -- need to fix benches.
kianenigma Dec 8, 2020
a7c4841
Some Staking stuff left to do.
kianenigma Dec 9, 2020
2ca3dcf
Master.into()
kianenigma Dec 14, 2020
4922897
Split the snapshot; type for CompactSolution and MaxNominations
kianenigma Dec 15, 2020
ef593a7
Fix a bunch more issues.
kianenigma Dec 15, 2020
8661814
check new design
kianenigma Dec 18, 2020
57f7294
Everything within the module works and compiles.
kianenigma Dec 21, 2020
af304d1
Stkaing also builds
kianenigma Dec 21, 2020
80e7450
Everything builds with the new scheme.
kianenigma Dec 25, 2020
4b4712a
Master.into()
kianenigma Dec 26, 2020
206933a
Simplify elect API.
kianenigma Dec 26, 2020
af9b97d
Fix all other tests.
kianenigma Dec 26, 2020
9663fea
make the staking api better and well tested.
kianenigma Jan 4, 2021
7f4f5b7
Almost done with conversion
kianenigma Jan 4, 2021
8fb11f1
Move to pallet V2 + construct_runtime
kianenigma Jan 5, 2021
bd6d4d9
Master.into()
kianenigma Jan 5, 2021
65f5ac1
Revert staking changes;
kianenigma Jan 5, 2021
3860885
Revert Staking fuzzer.
kianenigma Jan 5, 2021
392f35a
Fix fuzzers
kianenigma Jan 5, 2021
1337e1b
Update frame/election-providers/src/two_phase/mod.rs
kianenigma Jan 6, 2021
4f6ec0d
a round of self review
kianenigma Jan 6, 2021
b3113f7
A large number of bugfixes after local test and self-review
kianenigma Jan 7, 2021
abca039
Upstream.into()
kianenigma Jan 7, 2021
6bc578a
Master.into()
kianenigma Jan 7, 2021
876f666
Apply suggestions from code review
kianenigma Jan 7, 2021
81df1df
Update frame/election-providers/src/two_phase/benchmarking.rs
kianenigma Jan 7, 2021
e3862ce
Fix some review grumbles.
kianenigma Jan 8, 2021
ca06f8e
Fix some docs
kianenigma Jan 8, 2021
6206c32
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 8, 2021
95d7a3c
Master.into()
kianenigma Jan 8, 2021
63776ff
Maybe relocalte the code?
kianenigma Jan 8, 2021
4732854
Master.into()
kianenigma Jan 8, 2021
a922f0b
Witness -> SoltionSize
kianenigma Jan 8, 2021
6d8247a
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
e9f4437
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
02069e1
Update frame/session/src/lib.rs
kianenigma Jan 12, 2021
000ecdc
Revamp macros
kianenigma Jan 12, 2021
cb6f156
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
df04ef3
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
ee2a316
Master.into()
kianenigma Jan 12, 2021
0d92ed5
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
88a96a0
Fix build
kianenigma Jan 12, 2021
6480ce1
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
8a68915
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 12, 2021
f4e2c03
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 12, 2021
c41ecb0
fix all the tests.
kianenigma Jan 13, 2021
b42336a
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 13, 2021
f5e23f9
Fix the strange issue with cargo lock
kianenigma Jan 13, 2021
56bb93c
revamp closures.
kianenigma Jan 14, 2021
257795c
Update frame/two-phase-election-provider/src/benchmarking.rs
kianenigma Jan 14, 2021
9c114bc
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 14, 2021
1ac97f5
Some review grumbles.
kianenigma Jan 14, 2021
366b15b
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 14, 2021
afa5507
Upstream.into()
kianenigma Jan 14, 2021
5d206ab
Fix build
kianenigma Jan 14, 2021
ea193c7
Bring back benches
kianenigma Jan 14, 2021
60e57d7
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 14, 2021
77f53a3
Add a bunch of more fixes: signed weight check, tests etc.
kianenigma Jan 15, 2021
9ce9d6b
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 15, 2021
b30dfb3
Fix build
kianenigma Jan 15, 2021
6001d7d
Master.into()
kianenigma Jan 15, 2021
a066e4a
Last small tweaks
kianenigma Jan 15, 2021
10f6e7b
Fix all tests.
kianenigma Jan 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Some review grumbles.
  • Loading branch information
kianenigma committed Jan 14, 2021
commit 1ac97f5c69cb77dd65429efaf4faaee019e04744
4 changes: 3 additions & 1 deletion bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,8 @@ parameter_types! {
pub const SignedDepositByte: Balance = 1 * CENTS;

// fallback: no need to do on-chain phragmen initially.
pub const Fallback: pallet_two_phase_election_provider::FallbackStrategy = pallet_two_phase_election_provider::FallbackStrategy::Nothing;
pub const Fallback: pallet_two_phase_election_provider::FallbackStrategy =
pallet_two_phase_election_provider::FallbackStrategy::Nothing;

// unsigned configs
pub const TwoPhaseUnsignedPriority: TransactionPriority = StakingUnsignedPriority::get() - 1u64;
Expand Down Expand Up @@ -549,6 +550,7 @@ impl pallet_two_phase_election_provider::Config for Runtime {
type CompactSolution = pallet_staking::CompactAssignments;
type Fallback = Fallback;
type WeightInfo = pallet_two_phase_election_provider::weights::SubstrateWeight<Runtime>;
type BenchmarkingConfig = ();
}

parameter_types! {
Expand Down
2 changes: 1 addition & 1 deletion frame/staking/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ static_assertions = "1.1.0"
serde = { version = "1.0.101", optional = true }
codec = { package = "parity-scale-codec", version = "1.3.6", default-features = false, features = ["derive"] }
sp-std = { version = "2.0.0", default-features = false, path = "../../primitives/std" }
# TODO: TWO_PHASE:: ideally we should be able to get rid of this.
# TWO_PHASE_NOTE:: ideally we should be able to get rid of this.
sp-npos-elections = { version = "2.0.0", default-features = false, path = "../../primitives/npos-elections" }
sp-io ={ version = "2.0.0", default-features = false, path = "../../primitives/io" }
sp-runtime = { version = "2.0.0", default-features = false, path = "../../primitives/runtime" }
Expand Down
21 changes: 10 additions & 11 deletions frame/staking/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1077,43 +1077,42 @@ decl_storage! {
/// The last planned session scheduled by the session pallet.
///
/// This is basically in sync with the call to [`SessionManager::new_session`].
/// TODO: TWO_PHASE: needs care to set the initial value upon migration.
pub CurrentPlannedSession get(fn current_planned_session): SessionIndex;

/// Snapshot of validators at the beginning of the current election window. This should only
/// have a value when [`EraElectionStatus`] == `ElectionStatus::Open(_)`.
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub SnapshotValidators get(fn snapshot_validators): Option<Vec<T::AccountId>>;

/// Snapshot of nominators at the beginning of the current election window. This should only
/// have a value when [`EraElectionStatus`] == `ElectionStatus::Open(_)`.
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub SnapshotNominators get(fn snapshot_nominators): Option<Vec<T::AccountId>>;

/// The next validator set. At the end of an era, if this is available (potentially from the
/// result of an offchain worker), it is immediately used. Otherwise, the on-chain election
/// is executed.
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub QueuedElected get(fn queued_elected): Option<ElectionResult<T::AccountId, BalanceOf<T>>>;

/// The score of the current [`QueuedElected`].
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub QueuedScore get(fn queued_score): Option<ElectionScore>;

/// Flag to control the execution of the offchain election. When `Open(_)`, we accept
/// solutions to be submitted.
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub EraElectionStatus get(fn era_election_status): ElectionStatus<T::BlockNumber>;

/// True if the current **planned** session is final. Note that this does not take era
/// forcing into account.
///
/// TODO: TWO_PHASE: should be removed once we switch to two-phase.
/// TWO_PHASE_NOTE: should be removed once we switch to two-phase.
pub IsCurrentSessionFinal get(fn is_current_session_final): bool = false;

/// True if network has been upgraded to this version.
Expand Down Expand Up @@ -3028,7 +3027,7 @@ impl<T: Config> Module<T> {
///
/// Returns `Err(())` if less than [`MinimumValidatorCount`] validators have been elected, `Ok`
/// otherwise.
#[allow(dead_code)] // TODO: TWO_PHASE
#[allow(dead_code)] // TWO_PHASE_NOTE
pub fn process_election(
flat_supports: sp_npos_elections::Supports<T::AccountId>,
current_era: EraIndex,
Expand Down Expand Up @@ -3076,7 +3075,7 @@ impl<T: Config> Module<T> {
}

// emit event
// TODO: TWO_PHASE: remove the inner value.
// TWO_PHASE_NOTE: remove the inner value.
Self::deposit_event(RawEvent::StakingElection(ElectionCompute::Signed));

log!(
Expand All @@ -3099,7 +3098,7 @@ impl<T: Config> Module<T> {
"Experimental election provider outputted {:?}",
outcome
);
// TODO: TWO_PHASE: This code path shall not return anything for now. Later on, redirect the
// TWO_PHASE_NOTE: This code path shall not return anything for now. Later on, redirect the
// results to `process_election`.
None
}
Expand Down Expand Up @@ -3578,7 +3577,7 @@ where
}

fn can_report() -> bool {
// TODO: TWO_PHASE: we can get rid of this API
// TWO_PHASE_NOTE: we can get rid of this API
Self::era_election_status().is_closed()
}
}
Expand Down
2 changes: 0 additions & 2 deletions frame/staking/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5036,7 +5036,6 @@ mod election_data_provider {
45
);
assert_eq!(staking_events().len(), 1);
// TODO: TWO_PHASE: remove the internal value.
assert_eq!(*staking_events().last().unwrap(), RawEvent::StakingElection(ElectionCompute::OnChain));

for b in 21..45 {
Expand All @@ -5054,7 +5053,6 @@ mod election_data_provider {
70
);
assert_eq!(staking_events().len(), 3);
// TODO: TWO_PHASE: remove the internal value.
assert_eq!(*staking_events().last().unwrap(), RawEvent::StakingElection(ElectionCompute::OnChain));
})
}
Expand Down
30 changes: 13 additions & 17 deletions frame/two-phase-election-provider/src/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ benchmarks! {

finalize_signed_phase_reject_solution {
let receiver = account("receiver", 0, SEED);
let initial_balance = T::Currency::minimum_balance() * 10u32.into();
let initial_balance = T::Currency::minimum_balance().max(One::one()) * 10u32.into();
let deposit: BalanceOf<T> = 10u32.into();
T::Currency::make_free_balance_be(&receiver, initial_balance);
assert_ok!(T::Currency::reserve(&receiver, deposit));
Expand All @@ -243,14 +243,14 @@ benchmarks! {
let c in 1 .. (T::MaxSignedSubmissions::get() - 1);

// the solution will be worse than all of them meaning the score need to be checked against all.
let solution = RawSolution { score: [(1000_0000u128 - 1).into(), 0, 0], ..Default::default() };
let solution = RawSolution { score: [(1_0000_000u128 - 1).into(), 0, 0], ..Default::default() };

<CurrentPhase<T>>::put(Phase::Signed);
<Round<T>>::put(1);

for i in 0..c {
<SignedSubmissions<T>>::mutate(|queue| {
let solution = RawSolution { score: [(1000_0000 + i).into(), 0, 0], ..Default::default() };
let solution = RawSolution { score: [(1_0000_000 + i).into(), 0, 0], ..Default::default() };
let signed_submission = SignedSubmission { solution, ..Default::default() };
// note: this is quite tricky: we know that the queue will stay sorted here. The
// last will be best.
Expand All @@ -268,14 +268,14 @@ benchmarks! {

submit_unsigned {
// number of votes in snapshot.
let v in 2000 .. 3000;
let v in 2000 .. T::BenchmarkingConfig::VOTERS[1];
// number of targets in snapshot.
let t in 500 .. 800;
let t in 500 .. T::BenchmarkingConfig::TARGETS[1];
// number of assignments, i.e. compact.len(). This means the active nominators, thus must be
// a subset of `v` component.
let a in 500 .. 1500;
let a in 500 .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
// number of desired targets. Must be a subset of `t` component.
let d in 200 .. 400;
let d in 200 .. T::BenchmarkingConfig::DESIRED_TARGETS[1];

let witness = SolutionSize { voters: v, targets: t };
let raw_solution = solution_with_size::<T>(witness, a, d);
Expand All @@ -290,14 +290,14 @@ benchmarks! {
// This is checking a valid solution. The worse case is indeed a valid solution.
feasibility_check {
// number of votes in snapshot.
let v in 2000 .. 3000;
let v in 2000 .. T::BenchmarkingConfig::VOTERS[1];
// number of targets in snapshot.
let t in 500 .. 800;
let t in 500 .. T::BenchmarkingConfig::TARGETS[1];
// number of assignments, i.e. compact.len(). This means the active nominators, thus must be
// a subset of `v` component.
let a in 500 .. 1500;
let a in 500 .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
// number of desired targets. Must be a subset of `t` component.
let d in 200 .. 400;
let d in 200 .. T::BenchmarkingConfig::DESIRED_TARGETS[1];

let size = SolutionSize { voters: v, targets: t };
let raw_solution = solution_with_size::<T>(size, a, d);
Expand Down Expand Up @@ -341,15 +341,11 @@ mod test {
});

ExtBuilder::default().build_and_execute(|| {
assert_ok!(test_benchmark_finalize_signed_phase_accept_solution::<
Runtime,
>());
assert_ok!(test_benchmark_finalize_signed_phase_accept_solution::<Runtime>());
});

ExtBuilder::default().build_and_execute(|| {
assert_ok!(test_benchmark_finalize_signed_phase_reject_solution::<
Runtime,
>());
assert_ok!(test_benchmark_finalize_signed_phase_reject_solution::<Runtime>());
});

ExtBuilder::default().build_and_execute(|| {
Expand Down
35 changes: 30 additions & 5 deletions frame/two-phase-election-provider/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,25 @@ where
type DataProvider = T::DataProvider;
}

/// Configuration for the benchmarks of the pallet.
pub trait BenchmarkingConfig {
/// Range of voters.
const VOTERS: [u32; 2];
/// Range of targets.
const TARGETS: [u32; 2];
/// Range of active voters.
const ACTIVE_VOTERS: [u32; 2];
/// Range of desired targets.
const DESIRED_TARGETS: [u32; 2];
}

impl BenchmarkingConfig for () {
const VOTERS: [u32; 2] = [2000, 3000];
const TARGETS: [u32; 2] = [500, 800];
const ACTIVE_VOTERS: [u32; 2] = [500, 1500];
const DESIRED_TARGETS: [u32; 2] = [200, 400];
}

/// Current phase of the pallet.
#[derive(PartialEq, Eq, Clone, Copy, Encode, Decode, RuntimeDebug)]
pub enum Phase<Bn> {
Expand Down Expand Up @@ -580,6 +599,9 @@ pub mod pallet {
/// Configuration for the fallback
type Fallback: Get<FallbackStrategy>;

/// The configuration of benchmarking.
type BenchmarkingConfig: BenchmarkingConfig;

/// The weight of the pallet.
type WeightInfo: WeightInfo;
}
Expand Down Expand Up @@ -710,10 +732,8 @@ pub mod pallet {

// ensure solution claims is better.
let mut signed_submissions = Self::signed_submissions();
let maybe_index =
Self::insert_submission(&who, &mut signed_submissions, solution, size);
ensure!(maybe_index.is_some(), Error::<T>::QueueFull);
let index = maybe_index.expect("Option checked to be `Some`; qed.");
let index = Self::insert_submission(&who, &mut signed_submissions, solution, size)
.ok_or(Error::<T>::QueueFull)?;

// collect deposit. Thereafter, the function cannot fail.
// Defensive -- index is valid.
Expand All @@ -723,8 +743,12 @@ pub mod pallet {
.unwrap_or_default();
T::Currency::reserve(&who, deposit).map_err(|_| Error::<T>::CannotPayDeposit)?;

// store the new signed submission.
// Remove the weakest, if needed.
if signed_submissions.len() as u32 > T::MaxSignedSubmissions::get() {
Self::remove_weakest(&mut signed_submissions);
}
debug_assert!(signed_submissions.len() as u32 <= T::MaxSignedSubmissions::get());

log!(
info,
"queued signed solution with (claimed) score {:?}",
Expand All @@ -734,6 +758,7 @@ pub mod pallet {
.unwrap_or_default()
);

// store the new signed submission.
<SignedSubmissions<T>>::put(signed_submissions);
Self::deposit_event(Event::SolutionStored(ElectionCompute::Signed));
Ok(None.into())
Expand Down
1 change: 1 addition & 0 deletions frame/two-phase-election-provider/src/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,7 @@ impl crate::Config for Runtime {
type UnsignedPriority = UnsignedPriority;
type DataProvider = StakingMock;
type WeightInfo = DualMockWeightInfo;
type BenchmarkingConfig = ();
type OnChainAccuracy = Perbill;
type Fallback = Fallback;
type CompactSolution = TestCompact;
Expand Down
21 changes: 3 additions & 18 deletions frame/two-phase-election-provider/src/signed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,6 @@ where
/// solution quality. If insertion was successful, `Some(index)` is returned where index is the
/// index of the newly inserted item.
///
/// The length of the queue will always be kept less than or equal to `T::MaxSignedSubmissions`.
///
/// If a solution is removed, their bond is unreserved.
///
/// Invariant: The returned index is always a valid index in `queue` and can safely be used to
/// inspect the newly inserted element.
pub fn insert_submission(
Expand Down Expand Up @@ -184,23 +180,12 @@ where
// It is either 0 (in which case `0 <= queue.len()`) or one of the queue indices
// + 1. The biggest queue index is `queue.len() - 1`, thus `at <= queue.len()`.
queue.insert(at, submission);
// if length has exceeded the limit, eject the weakest, return shifted index.
if queue.len() as u32 > T::MaxSignedSubmissions::get() {
Self::remove_weakest(queue);
// Invariant: at > 0
// Proof by contradiction: Assume at is 0 and this will underflow. Then the
// previous length of the queue must have been `T::MaxSignedSubmissions` so
// that `queue.len() + 1` (for the newly inserted one) is more than
// `T::MaxSignedSubmissions`. But this would have been detected by the first
// if branch; qed.
Some(at - 1)
} else {
Some(at)
}
Some(at)
}
});

debug_assert!(queue.len() as u32 <= T::MaxSignedSubmissions::get());
// If the call site is sane and removes the weakest, then this must always be correct.
debug_assert!(queue.len() as u32 <= T::MaxSignedSubmissions::get() + 1);
outcome
}

Expand Down