Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
25bddc9
mockup
kianenigma Sep 11, 2020
7bef52f
Add files
kianenigma Sep 11, 2020
4f3dfa4
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Sep 23, 2020
bc30ce1
More mockup
kianenigma Sep 24, 2020
9e52fbc
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 1, 2020
14ebc95
Lots more changes and tests. Initial design of signed is forseable now.
kianenigma Oct 2, 2020
0587fc9
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 8, 2020
233d4e1
More changes for the signed phase, now going to the cluster-fu*k of g…
kianenigma Oct 9, 2020
9c880f3
Finally decoupled solution from the pallet.. such a PITA.
kianenigma Oct 9, 2020
0cb35eb
Much simplifications and cleanups.
kianenigma Oct 9, 2020
c1ff3ec
A skeleton for the unsigned version.
kianenigma Oct 9, 2020
85a346e
Master.into()
kianenigma Oct 13, 2020
6a2542d
Staking builds and tests compile, some tests fail
kianenigma Oct 13, 2020
c361911
All staking tests work
kianenigma Oct 14, 2020
218db10
Unsigned almost done, primitive crate added.
kianenigma Oct 15, 2020
5e21a79
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Oct 15, 2020
25c1bc3
Use decl_error
kianenigma Oct 15, 2020
d53785c
Add basic benchmark setup, needs more work
kianenigma Oct 16, 2020
525f6e0
Comment.
kianenigma Oct 16, 2020
3026316
Checkpoint.
kianenigma Oct 20, 2020
9841e3b
node-runtime builds.
kianenigma Oct 20, 2020
16c0550
Builds and all good?
kianenigma Dec 3, 2020
f197b6b
Undo some formatting mistakes
kianenigma Dec 3, 2020
5005281
Working on benchmarks'
kianenigma Dec 7, 2020
bd0d331
Fix bench
kianenigma Dec 8, 2020
67dd4d1
Strange merge -- everything builds.
kianenigma Dec 8, 2020
eac4571
Everything compiles and good -- need to fix benches.
kianenigma Dec 8, 2020
a7c4841
Some Staking stuff left to do.
kianenigma Dec 9, 2020
2ca3dcf
Master.into()
kianenigma Dec 14, 2020
4922897
Split the snapshot; type for CompactSolution and MaxNominations
kianenigma Dec 15, 2020
ef593a7
Fix a bunch more issues.
kianenigma Dec 15, 2020
8661814
check new design
kianenigma Dec 18, 2020
57f7294
Everything within the module works and compiles.
kianenigma Dec 21, 2020
af304d1
Stkaing also builds
kianenigma Dec 21, 2020
80e7450
Everything builds with the new scheme.
kianenigma Dec 25, 2020
4b4712a
Master.into()
kianenigma Dec 26, 2020
206933a
Simplify elect API.
kianenigma Dec 26, 2020
af9b97d
Fix all other tests.
kianenigma Dec 26, 2020
9663fea
make the staking api better and well tested.
kianenigma Jan 4, 2021
7f4f5b7
Almost done with conversion
kianenigma Jan 4, 2021
8fb11f1
Move to pallet V2 + construct_runtime
kianenigma Jan 5, 2021
bd6d4d9
Master.into()
kianenigma Jan 5, 2021
65f5ac1
Revert staking changes;
kianenigma Jan 5, 2021
3860885
Revert Staking fuzzer.
kianenigma Jan 5, 2021
392f35a
Fix fuzzers
kianenigma Jan 5, 2021
1337e1b
Update frame/election-providers/src/two_phase/mod.rs
kianenigma Jan 6, 2021
4f6ec0d
a round of self review
kianenigma Jan 6, 2021
b3113f7
A large number of bugfixes after local test and self-review
kianenigma Jan 7, 2021
abca039
Upstream.into()
kianenigma Jan 7, 2021
6bc578a
Master.into()
kianenigma Jan 7, 2021
876f666
Apply suggestions from code review
kianenigma Jan 7, 2021
81df1df
Update frame/election-providers/src/two_phase/benchmarking.rs
kianenigma Jan 7, 2021
e3862ce
Fix some review grumbles.
kianenigma Jan 8, 2021
ca06f8e
Fix some docs
kianenigma Jan 8, 2021
6206c32
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 8, 2021
95d7a3c
Master.into()
kianenigma Jan 8, 2021
63776ff
Maybe relocalte the code?
kianenigma Jan 8, 2021
4732854
Master.into()
kianenigma Jan 8, 2021
a922f0b
Witness -> SoltionSize
kianenigma Jan 8, 2021
6d8247a
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
e9f4437
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 8, 2021
02069e1
Update frame/session/src/lib.rs
kianenigma Jan 12, 2021
000ecdc
Revamp macros
kianenigma Jan 12, 2021
cb6f156
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
df04ef3
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
ee2a316
Master.into()
kianenigma Jan 12, 2021
0d92ed5
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 12, 2021
88a96a0
Fix build
kianenigma Jan 12, 2021
6480ce1
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 12, 2021
8a68915
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 12, 2021
f4e2c03
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 12, 2021
c41ecb0
fix all the tests.
kianenigma Jan 13, 2021
b42336a
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 13, 2021
f5e23f9
Fix the strange issue with cargo lock
kianenigma Jan 13, 2021
56bb93c
revamp closures.
kianenigma Jan 14, 2021
257795c
Update frame/two-phase-election-provider/src/benchmarking.rs
kianenigma Jan 14, 2021
9c114bc
Merge remote-tracking branch 'origin/master' into kiz-election-provider
Jan 14, 2021
1ac97f5
Some review grumbles.
kianenigma Jan 14, 2021
366b15b
Merge branch 'master' of github.com:paritytech/substrate into kiz-ele…
kianenigma Jan 14, 2021
afa5507
Upstream.into()
kianenigma Jan 14, 2021
5d206ab
Fix build
kianenigma Jan 14, 2021
ea193c7
Bring back benches
kianenigma Jan 14, 2021
60e57d7
cargo run --release --features=runtime-benchmarks --manifest-path=bin…
Jan 14, 2021
77f53a3
Add a bunch of more fixes: signed weight check, tests etc.
kianenigma Jan 15, 2021
9ce9d6b
Merge branch 'kiz-election-provider' of github.com:paritytech/substra…
kianenigma Jan 15, 2021
b30dfb3
Fix build
kianenigma Jan 15, 2021
6001d7d
Master.into()
kianenigma Jan 15, 2021
a066e4a
Last small tweaks
kianenigma Jan 15, 2021
10f6e7b
Fix all tests.
kianenigma Jan 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Finally decoupled solution from the pallet.. such a PITA.
  • Loading branch information
kianenigma committed Oct 9, 2020
commit 9c880f3dc2ec18e7278fe02b85fe746f36b97e70
41 changes: 12 additions & 29 deletions frame/election-providers/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,15 @@

#![cfg_attr(not(feature = "std"), no_std)]

use sp_std::prelude::*;
use sp_std::{fmt::Debug, prelude::*};

/// The onchain module.
pub mod onchain;
/// The two-phase module.
pub mod two_phase;

use sp_arithmetic::PerThing;
use sp_npos_elections::{ExtendedBalance, Support, SupportMap};
use sp_npos_elections::{CompactSolution, ExtendedBalance, PerThing128, Support, SupportMap};

// for the helper macros
#[doc(hidden)]
Expand Down Expand Up @@ -87,7 +88,11 @@ impl<A> FlattenSupportMap<A> for SupportMap<A> {
/// }
/// ```
pub trait ElectionDataProvider<AccountId, B> {
type CompactSolution: codec::Codec + Default + PartialEq + Eq;
/// The compact solution type.
///
/// This should encode the entire solution with the least possible space usage.
type CompactSolution: codec::Codec + Default + PartialEq + Eq + Clone + Debug + CompactSolution;

/// All possible targets for the election, i.e. the candidates.
fn targets() -> Vec<AccountId>;

Expand Down Expand Up @@ -116,25 +121,6 @@ pub trait ElectionDataProvider<AccountId, B> {
fn next_election_prediction(now: B) -> B;
}

#[cfg(feature = "std")]
impl<AccountId, B: Default> ElectionDataProvider<AccountId, B> for () {
fn targets() -> Vec<AccountId> {
Default::default()
}
fn voters() -> Vec<(AccountId, VoteWeight, Vec<AccountId>)> {
Default::default()
}
fn desired_targets() -> u32 {
Default::default()
}
fn feasibility_check_assignment<P: PerThing>(_: &AccountId, _: &[(AccountId, P)]) -> bool {
Default::default()
}
fn next_election_prediction(_: B) -> B {
Default::default()
}
}

/// Something that can compute the result of an election and pass it back to the caller.
pub trait ElectionProvider<AccountId> {
/// Indicate weather this election provider needs data when calling [`elect`] or not.
Expand All @@ -150,18 +136,15 @@ pub trait ElectionProvider<AccountId> {
/// Note that based on the logic of the type that will implement this trait, the input data may
/// or may not be used. To hint about this to the call site, [`NEEDS_ELECT_DATA`] should be
/// properly set.
fn elect<P: PerThing>(
///
/// The implementation should, if possible, use the accuracy `P` to compute the election result.
fn elect<P: PerThing128>(
to_elect: usize, // TODO: consider making this u32
targets: Vec<AccountId>,
voters: Vec<(AccountId, VoteWeight, Vec<AccountId>)>,
) -> Result<FlatSupportMap<AccountId>, Self::Error>
where
// TODO: Okay about these two, I get that we probably need the first one, but can't we
// alleviate the latter one? I think we can say that all PerThing are Mul of some types.
// Perhaps it is time to move the PerBill macros to something better? Yeah I think then we
// can get rid of both of these types everywhere.
ExtendedBalance: From<<P as PerThing>::Inner>,
P: sp_std::ops::Mul<ExtendedBalance, Output = ExtendedBalance>;
ExtendedBalance: From<<P as PerThing>::Inner>;

/// Returns true if an election is still ongoing. This can be used by the call site to
/// dynamically check of a long-lasting election (such as [`two_phase`]) is still on-going or
Expand Down
6 changes: 3 additions & 3 deletions frame/election-providers/src/onchain.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::{ElectionProvider, FlatSupportMap, FlattenSupportMap};
use sp_arithmetic::PerThing;
use sp_npos_elections::{ElectionResult, ExtendedBalance, IdentifierT, VoteWeight};
use sp_npos_elections::{ElectionResult, ExtendedBalance, IdentifierT, PerThing128, VoteWeight};
use sp_runtime::RuntimeDebug;
use sp_std::{collections::btree_map::BTreeMap, prelude::*};

Expand All @@ -23,14 +23,13 @@ impl<AccountId: IdentifierT> ElectionProvider<AccountId> for OnChainSequentialPh

const NEEDS_ELECT_DATA: bool = true;

fn elect<P: sp_arithmetic::PerThing>(
fn elect<P: PerThing128>(
to_elect: usize,
targets: Vec<AccountId>,
voters: Vec<(AccountId, VoteWeight, Vec<AccountId>)>,
) -> Result<FlatSupportMap<AccountId>, Self::Error>
where
ExtendedBalance: From<<P as PerThing>::Inner>,
P: sp_std::ops::Mul<ExtendedBalance, Output = ExtendedBalance>,
{
// TODO: we really don't need to do this conversion all the time. With
// https://github.com/paritytech/substrate/pull/6685 merged, we should make variants of
Expand All @@ -39,6 +38,7 @@ impl<AccountId: IdentifierT> ElectionProvider<AccountId> for OnChainSequentialPh
// TODO: Okay even if not the above, then def. make the extension traits for converting
// between validator Major and Nominator Major result types, and make the conversions be
// lossless and painless to happen.

let mut stake_map: BTreeMap<AccountId, VoteWeight> = BTreeMap::new();
voters.iter().for_each(|(v, s, _)| {
stake_map.insert(v.clone(), *s);
Expand Down
20 changes: 10 additions & 10 deletions frame/election-providers/src/two_phase/macros.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,25 +29,25 @@ macro_rules! log {

#[macro_export]
macro_rules! voter_index_fn {
($voters:ident, $acc:ty) => {
|who: &$acc| -> Option<$crate::two_phase::VoterIndex> {
($voters:ident, $acc:ty, $t:ident) => {
|who: &$acc| -> Option<$crate::two_phase::CompactVoterIndexOf<$t>> {
$voters
.iter()
.position(|(x, _, _)| x == who)
.and_then(|i| <usize as $crate::TryInto<$crate::two_phase::VoterIndex>>::try_into(i).ok())
.and_then(|i| <usize as $crate::TryInto<$crate::two_phase::CompactVoterIndexOf<$t>>>::try_into(i).ok())
}
};
}

#[macro_export]
macro_rules! target_index_fn {
($targets:ident, $acc:ty) => {
|who: &$acc| -> Option<$crate::two_phase::TargetIndex> {
$targets
.iter()
.position(|x| x == who)
.and_then(|i| <usize as $crate::TryInto<$crate::two_phase::TargetIndex>>::try_into(i).ok())
}
($targets:ident, $acc:ty, $t:ident) => {
|who: &$acc| -> Option<$crate::two_phase::CompactTargetIndexOf<$t>> {
$targets
.iter()
.position(|x| x == who)
.and_then(|i| <usize as $crate::TryInto<$crate::two_phase::CompactTargetIndexOf<$t>>>::try_into(i).ok())
}
};
}

Expand Down
36 changes: 20 additions & 16 deletions frame/election-providers/src/two_phase/mock.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
use super::*;
use frame_support::{parameter_types, traits::OnInitialize};
use sp_core::H256;
use sp_npos_elections::CompactSolution;
use sp_runtime::{
testing::Header,
traits::{BlakeTwo256, IdentityLookup},
PerU16,
};
use std::cell::RefCell;

Expand All @@ -17,6 +19,11 @@ pub(crate) type TwoPhase = super::Module<Runtime>;
pub(crate) type Balance = u64;
pub(crate) type AccountId = u64;

sp_npos_elections::generate_solution_type!(
#[compact]
pub struct TestCompact::<u32, u16, PerU16>(16)
);

/// To from `now` to block `n`.
pub fn roll_to(n: u64) {
let now = System::block_number();
Expand All @@ -34,21 +41,22 @@ pub fn balances(who: &AccountId) -> (Balance, Balance) {
/// Spit out a verifiable raw solution.
///
/// This is a good example of what an offchain miner would do.
pub fn raw_solution() -> RawSolution {
pub fn raw_solution() -> RawSolution<CompactOf<Runtime>> {
let voters = TwoPhase::snapshot_voters().unwrap();
let targets = TwoPhase::snapshot_targets().unwrap();
let desired = TwoPhase::desired_targets() as usize;

// closures
let voter_index = crate::voter_index_fn!(voters, AccountId);
let target_index = crate::target_index_fn!(targets, AccountId);
let voter_index = crate::voter_index_fn!(voters, AccountId, Runtime);
let target_index = crate::target_index_fn!(targets, AccountId, Runtime);
let stake_of = crate::stake_of_fn!(voters, AccountId);

use sp_npos_elections::{seq_phragmen, to_without_backing, ElectionResult};
let ElectionResult {
winners,
assignments,
} = seq_phragmen::<_, OffchainAccuracy>(desired, targets.clone(), voters.clone(), None).unwrap();
} = seq_phragmen::<_, CompactAccuracyOf<Runtime>>(desired, targets.clone(), voters.clone(), None)
.unwrap();

let winners = to_without_backing(winners);

Expand All @@ -59,17 +67,9 @@ pub fn raw_solution() -> RawSolution {
sp_npos_elections::evaluate_support(&support)
};
let compact =
CompactAssignments::from_assignment(assignments, &voter_index, &target_index).unwrap();
let winners = winners
.into_iter()
.map(|w| target_index(&w).unwrap())
.collect::<Vec<_>>();
<CompactOf<Runtime>>::from_assignment(assignments, &voter_index, &target_index).unwrap();

RawSolution {
winners,
compact,
score,
}
RawSolution { compact, score }
}

frame_support::impl_outer_origin! {
Expand Down Expand Up @@ -191,7 +191,7 @@ impl crate::two_phase::Trait for Runtime {
type SolutionImprovementThreshold = ();
type SlashHandler = ();
type RewardHandler = ();
type ElectionDataProvider = ExtBuilder;
type ElectionDataProvider = StakingMock;
type WeightInfo = ();
}

Expand All @@ -207,7 +207,11 @@ impl Default for ExtBuilder {
}
}

impl crate::ElectionDataProvider<AccountId, u64> for ExtBuilder {
pub struct StakingMock;

impl crate::ElectionDataProvider<AccountId, u64> for StakingMock {
type CompactSolution = TestCompact;

fn targets() -> Vec<AccountId> {
Targets::get()
}
Expand Down
Loading