Skip to content

Commit 764018b

Browse files
cmichiMTDK1
authored andcommitted
Adjust consensus telemetry (paritytech#2198)
* Send high-level consensus telemetry by default * Notify telemetry on finalized * Send used authority set to telemetry * Do not send commit message telemetry by default * Fix typo * Allow for notifications on telemetry connect ...and send the current authority set on each connect. * Send authority set to telemetry on change * Revert "Send used authority set to telemetry" This reverts commit 1deceea. * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' Squashed commit of the following: commit 6de583a Author: Xiliang Chen <xlchen1291@gmail.com> Date: Wed Apr 10 20:26:29 2019 +1200 update authers for rest of the node-template cargo.toml files (paritytech#2242) commit 5240bc1 Author: Bastian Köcher <bkchr@users.noreply.github.com> Date: Tue Apr 9 10:31:18 2019 +0200 Throw a compile error for `on_finalise` and `on_initialise` (paritytech#2236) commit 67d2e71 Author: Pierre Krieger <pierre.krieger1708@gmail.com> Date: Tue Apr 9 05:30:43 2019 -0300 Add warning when using default protocol ID (paritytech#2234) * Add warning when using default protocol ID * Update core/service/src/lib.rs commit 1421fed Author: Xiliang Chen <xlchen1291@gmail.com> Date: Tue Apr 9 17:22:20 2019 +1200 update name and authors to placeholder text for node-template (paritytech#2222) * update name and authors to placeholder text * revert package name change commit 6617f23 Author: André Silva <andre.beat@gmail.com> Date: Mon Apr 8 12:50:34 2019 +0100 grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (paritytech#2139) * core: grandpa: migrate to grandpa 0.7 * core: grandpa: store current round votes and load them on startup * core: grandpa: resend old persisted votes for the current round * core: grandpa: store base and votes for last completed round * core: grandpa: fix latest grandpa 0.7 changes * core: grandpa: update to grandpa 0.7.1 * core: grandpa: persist votes for last two completed rounds * core: grandpa: simplify VoterSetState usage * core: grandpa: use Environment::update_voter_set_state * core: grandpa: fix aux_schema test * core: grandpa: add docs * core: grandpa: add note about environment assumption * core: grandpa: don't update voter set state on ignored votes * core: grandpa: add test for v1 -> v2 aux_schema migration * core: grandpa: add test for voter vote persistence * core: grandpa: use grandpa 0.7.1 from crates.io * core: grandpa: use try_init in test * core: grandpa: add comment about block_import in test * core: grandpa: avoid cloning HasVoted * core: grandpa: add missing docs * core: grandpa: cleanup up can_propose/prevote/precommit commit 21e0877 Author: Gregory Terzian <2792687+gterzian@users.noreply.github.com> Date: Mon Apr 8 13:17:00 2019 +0200 remove clone bound on specialization in testnet factory (paritytech#2157) commit 7c64746 Author: Andrew Jones <ascjones@gmail.com> Date: Sat Apr 6 12:23:56 2019 +0100 Contract import/export validation (paritytech#2203) * Reject validation of contract with unknown exports * Validate imports eagerly * Increment spec version commit 12718fa Author: Pierre Krieger <pierre.krieger1708@gmail.com> Date: Fri Apr 5 14:07:09 2019 -0300 Fix state inconsistency between handler and behaviour (paritytech#2220) * Fix state inconsistency between handler and behaviour * Fix the error! being in the wrong place commit f917d12 Author: Bastian Köcher <bkchr@users.noreply.github.com> Date: Fri Apr 5 18:50:38 2019 +0200 Use `storage_root` of newly calculated header (paritytech#2216) Instead of calculating the `storage_root` a second time, we just can take the `storage_root` from the new header. commit 3359ce0 Author: Marek Kotewicz <marek.kotewicz@gmail.com> Date: Fri Apr 5 14:44:46 2019 +0200 Peerset::discovered accepts many peer ids (paritytech#2213) * Peerset::discovered accepts many peer ids * Improve tracing in peerset commit dd82e0e Author: Marek Kotewicz <marek.kotewicz@gmail.com> Date: Thu Apr 4 19:40:40 2019 +0200 simplification of peerset api (paritytech#2123) * Introduction of PeersetHandle * integrate PeersetHandle with the rest of the codebase * fix compilation errors * more tests for peerset, fixed overwriting bug in add_reserved_peer * Slots data structure and bugfixes for peerset * bend to pressure * updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2 * peerset discovered list is now a LinkedHashMap * fix review suggestions * split back Peerset and PeersetHandle * test for Peerset::discovered * applied review suggestions * fixes to peerset::incoming * peerset disconnects are all instantaneous * instantaneous drop in peerset finished * Peerset::set_reserved_only can also reconnect nodes * Peerset scores cache uses lru-cache * remove redundant function call and comment from Peerset::on_set_reserved_only * add_peer returns SlotState enum * apply review suggestions * is_reserved -> is_connected_and_reserved commit d90833d Author: Arkadiy Paronyan <arkady.paronyan@gmail.com> Date: Thu Apr 4 18:01:28 2019 +0200 Disconnect on protocol timeout (paritytech#2212) commit c0a46b5 Author: André Silva <andre.beat@gmail.com> Date: Thu Apr 4 15:56:49 2019 +0100 core: grandpa: verify commit target in justification (paritytech#2201) commit 3a4901a Author: Bastian Köcher <bkchr@users.noreply.github.com> Date: Thu Apr 4 16:56:16 2019 +0200 Introduce `original_storage` and `original_storage_hash` (paritytech#2211) Both functions will ignore any overlayed changes and access the backend directly. commit a7a469f Author: Xiliang Chen <xlchen1291@gmail.com> Date: Fri Apr 5 03:55:55 2019 +1300 code cleanup (paritytech#2206) commit 26c7b44 Author: Arkadiy Paronyan <arkady.paronyan@gmail.com> Date: Wed Apr 3 15:52:46 2019 +0200 Emberic elm testnet (paritytech#2197) * Make telemetry onconnect hoook optional * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' * Introduce GrandpaParams struct to condense parameters * Remove debug statement * Fix tests * Rename parameter * Fix tests * Rename struct * Do not send verbosity level * Combine imports * Implement comments * Run cargo build --all * Remove noisy telemetry * Add docs for public items * Unbox and support Clone trait * Fix merge * Fix merge * Update core/finality-grandpa/src/lib.rs Co-Authored-By: cmichi <mich@elmueller.net>
1 parent 24c3aad commit 764018b

File tree

12 files changed

+166
-55
lines changed

12 files changed

+166
-55
lines changed

Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

core/client/src/client.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -956,6 +956,11 @@ impl<B, E, Block, RA> Client<B, E, Block, RA> where
956956
let header = self.header(&BlockId::Hash(finalized_hash))?
957957
.expect("header already known to exist in DB because it is indicated in the tree route; qed");
958958

959+
telemetry!(SUBSTRATE_INFO; "notify.finalized";
960+
"height" => format!("{}", header.number()),
961+
"best" => ?finalized_hash,
962+
);
963+
959964
let notification = FinalityNotification {
960965
header,
961966
hash: finalized_hash,

core/finality-grandpa/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ runtime_primitives = { package = "sr-primitives", path = "../sr-primitives" }
1616
consensus_common = { package = "substrate-consensus-common", path = "../consensus/common" }
1717
substrate-primitives = { path = "../primitives" }
1818
substrate-telemetry = { path = "../telemetry" }
19+
serde_json = "1.0"
1920
client = { package = "substrate-client", path = "../client" }
2021
inherents = { package = "substrate-inherents", path = "../../core/inherents" }
2122
network = { package = "substrate-network", path = "../network" }

core/finality-grandpa/src/aux_schema.rs

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ use fork_tree::ForkTree;
2525
use grandpa::round::State as RoundState;
2626
use runtime_primitives::traits::{Block as BlockT, NumberFor};
2727
use log::{info, warn};
28+
use substrate_telemetry::{telemetry, CONSENSUS_INFO};
2829

2930
use crate::authorities::{AuthoritySet, SharedAuthoritySet, PendingChange, DelayKind};
3031
use crate::consensus_changes::{SharedConsensusChanges, ConsensusChanges};
@@ -365,6 +366,17 @@ pub(crate) fn update_authority_set<Block: BlockT, F, R>(
365366
let encoded_set = set.encode();
366367

367368
if let Some(new_set) = new_set {
369+
telemetry!(CONSENSUS_INFO; "afg.authority_set";
370+
"hash" => ?new_set.canon_hash,
371+
"number" => ?new_set.canon_number,
372+
"authority_set_id" => ?new_set.set_id,
373+
"authorities" => {
374+
let authorities: Vec<String> =
375+
new_set.authorities.iter().map(|(id, _)| format!("{}", id)).collect();
376+
format!("{:?}", authorities)
377+
}
378+
);
379+
368380
// we also overwrite the "last completed round" entry with a blank slate
369381
// because from the perspective of the finality gadget, the chain has
370382
// reset.

core/finality-grandpa/src/communication/mod.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -681,7 +681,7 @@ impl<Block: BlockT, N: Network<Block>> Sink for CommitsOut<Block, N> {
681681
let (round, commit) = input;
682682
let round = Round(round);
683683

684-
telemetry!(CONSENSUS_INFO; "afg.commit_issued";
684+
telemetry!(CONSENSUS_DEBUG; "afg.commit_issued";
685685
"target_number" => ?commit.target_number, "target_hash" => ?commit.target_hash,
686686
);
687687
let (precommits, auth_data) = commit.precommits.into_iter()

core/finality-grandpa/src/lib.rs

Lines changed: 56 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ use inherents::InherentDataProviders;
6969
use runtime_primitives::generic::BlockId;
7070
use substrate_primitives::{ed25519, H256, Pair, Blake2Hasher};
7171
use substrate_telemetry::{telemetry, CONSENSUS_INFO, CONSENSUS_DEBUG, CONSENSUS_WARN};
72+
use serde_json;
7273

7374
use srml_finality_tracker;
7475

@@ -105,6 +106,7 @@ use environment::{CompletedRound, CompletedRounds, Environment, HasVoted, Shared
105106
use import::GrandpaBlockImport;
106107
use until_imported::UntilCommitBlocksImported;
107108
use communication::NetworkBridge;
109+
use service::TelemetryOnConnect;
108110

109111
use ed25519::{Public as AuthorityId, Signature as AuthoritySignature};
110112

@@ -433,14 +435,26 @@ fn register_finality_tracker_inherent_data_provider<B, E, Block: BlockT<Hash=H25
433435
}
434436
}
435437

438+
/// Parameters used to run Grandpa.
439+
pub struct GrandpaParams<'a, B, E, Block: BlockT<Hash=H256>, N, RA, X> {
440+
/// Configuration for the GRANDPA service.
441+
pub config: Config,
442+
/// A link to the block import worker.
443+
pub link: LinkHalf<B, E, Block, RA>,
444+
/// The Network instance.
445+
pub network: N,
446+
/// The inherent data providers.
447+
pub inherent_data_providers: InherentDataProviders,
448+
/// Handle to a future that will resolve on exit.
449+
pub on_exit: X,
450+
/// If supplied, can be used to hook on telemetry connection established events.
451+
pub telemetry_on_connect: Option<TelemetryOnConnect<'a>>,
452+
}
453+
436454
/// Run a GRANDPA voter as a task. Provide configuration and a link to a
437455
/// block import worker that has already been instantiated with `block_import`.
438-
pub fn run_grandpa_voter<B, E, Block: BlockT<Hash=H256>, N, RA>(
439-
config: Config,
440-
link: LinkHalf<B, E, Block, RA>,
441-
network: N,
442-
inherent_data_providers: InherentDataProviders,
443-
on_exit: impl Future<Item=(),Error=()> + Clone + Send + 'static,
456+
pub fn run_grandpa_voter<B, E, Block: BlockT<Hash=H256>, N, RA, X>(
457+
grandpa_params: GrandpaParams<B, E, Block, N, RA, X>,
444458
) -> ::client::error::Result<impl Future<Item=(),Error=()> + Send + 'static> where
445459
Block::Hash: Ord,
446460
B: Backend<Block, Blake2Hasher> + 'static,
@@ -451,7 +465,17 @@ pub fn run_grandpa_voter<B, E, Block: BlockT<Hash=H256>, N, RA>(
451465
DigestFor<Block>: Encode,
452466
DigestItemFor<Block>: DigestItem<AuthorityId=AuthorityId>,
453467
RA: Send + Sync + 'static,
468+
X: Future<Item=(),Error=()> + Clone + Send + 'static,
454469
{
470+
let GrandpaParams {
471+
config,
472+
link,
473+
network,
474+
inherent_data_providers,
475+
on_exit,
476+
telemetry_on_connect,
477+
} = grandpa_params;
478+
455479
use futures::future::{self, Loop as FutureLoop};
456480

457481
let (network, network_startup) = NetworkBridge::new(network, config.clone(), on_exit.clone());
@@ -465,6 +489,28 @@ pub fn run_grandpa_voter<B, E, Block: BlockT<Hash=H256>, N, RA>(
465489

466490
register_finality_tracker_inherent_data_provider(client.clone(), &inherent_data_providers)?;
467491

492+
if let Some(telemetry_on_connect) = telemetry_on_connect {
493+
let authorities = authority_set.clone();
494+
let events = telemetry_on_connect.telemetry_connection_sinks
495+
.for_each(move |_| {
496+
telemetry!(CONSENSUS_INFO; "afg.authority_set";
497+
"authority_set_id" => ?authorities.set_id(),
498+
"authorities" => {
499+
let curr = authorities.current_authorities();
500+
let voters = curr.voters();
501+
let authorities: Vec<String> =
502+
voters.iter().map(|(id, _)| id.to_string()).collect();
503+
serde_json::to_string(&authorities)
504+
.expect("authorities is always at least an empty vector; elements are always of type string")
505+
}
506+
);
507+
Ok(())
508+
})
509+
.then(|_| Ok(()));
510+
let events = events.select(telemetry_on_connect.on_exit).then(|_| Ok(()));
511+
telemetry_on_connect.executor.spawn(events);
512+
}
513+
468514
let voters = authority_set.current_authorities();
469515
let initial_environment = Arc::new(Environment {
470516
inner: client.clone(),
@@ -660,12 +706,8 @@ pub fn run_grandpa_voter<B, E, Block: BlockT<Hash=H256>, N, RA>(
660706
}
661707

662708
#[deprecated(since = "1.1", note = "Please switch to run_grandpa_voter.")]
663-
pub fn run_grandpa<B, E, Block: BlockT<Hash=H256>, N, RA>(
664-
config: Config,
665-
link: LinkHalf<B, E, Block, RA>,
666-
network: N,
667-
inherent_data_providers: InherentDataProviders,
668-
on_exit: impl Future<Item=(),Error=()> + Clone + Send + 'static,
709+
pub fn run_grandpa<B, E, Block: BlockT<Hash=H256>, N, RA, X>(
710+
grandpa_params: GrandpaParams<B, E, Block, N, RA, X>,
669711
) -> ::client::error::Result<impl Future<Item=(),Error=()> + Send + 'static> where
670712
Block::Hash: Ord,
671713
B: Backend<Block, Blake2Hasher> + 'static,
@@ -676,6 +718,7 @@ pub fn run_grandpa<B, E, Block: BlockT<Hash=H256>, N, RA>(
676718
DigestFor<Block>: Encode,
677719
DigestItemFor<Block>: DigestItem<AuthorityId=AuthorityId>,
678720
RA: Send + Sync + 'static,
721+
X: Future<Item=(),Error=()> + Clone + Send + 'static,
679722
{
680-
run_grandpa_voter(config, link, network, inherent_data_providers, on_exit)
723+
run_grandpa_voter(grandpa_params)
681724
}

core/finality-grandpa/src/tests.rs

Lines changed: 37 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -419,18 +419,20 @@ fn run_to_completion_with<F>(
419419

420420
fn assert_send<T: Send>(_: &T) { }
421421

422-
let voter = run_grandpa_voter(
423-
Config {
422+
let grandpa_params = GrandpaParams {
423+
config: Config {
424424
gossip_duration: TEST_GOSSIP_DURATION,
425425
justification_period: 32,
426426
local_key: Some(Arc::new(key.clone().into())),
427427
name: Some(format!("peer#{}", peer_id)),
428428
},
429-
link,
430-
MessageRouting::new(net.clone(), peer_id),
431-
InherentDataProviders::new(),
432-
Exit,
433-
).expect("all in order with client and network");
429+
link: link,
430+
network: MessageRouting::new(net.clone(), peer_id),
431+
inherent_data_providers: InherentDataProviders::new(),
432+
on_exit: Exit,
433+
telemetry_on_connect: None,
434+
};
435+
let voter = run_grandpa_voter(grandpa_params).expect("all in order with client and network");
434436

435437
assert_send(&voter);
436438

@@ -517,18 +519,21 @@ fn finalize_3_voters_1_full_observer() {
517519
.take_while(|n| Ok(n.header.number() < &20))
518520
.for_each(move |_| Ok(()))
519521
);
520-
let voter = run_grandpa_voter(
521-
Config {
522+
523+
let grandpa_params = GrandpaParams {
524+
config: Config {
522525
gossip_duration: TEST_GOSSIP_DURATION,
523526
justification_period: 32,
524527
local_key,
525528
name: Some(format!("peer#{}", peer_id)),
526529
},
527-
link,
528-
MessageRouting::new(net.clone(), peer_id),
529-
InherentDataProviders::new(),
530-
Exit,
531-
).expect("all in order with client and network");
530+
link: link,
531+
network: MessageRouting::new(net.clone(), peer_id),
532+
inherent_data_providers: InherentDataProviders::new(),
533+
on_exit: Exit,
534+
telemetry_on_connect: None,
535+
};
536+
let voter = run_grandpa_voter(grandpa_params).expect("all in order with client and network");
532537

533538
runtime.spawn(voter);
534539
}
@@ -679,18 +684,20 @@ fn transition_3_voters_twice_1_full_observer() {
679684
assert_eq!(set.pending_changes().count(), 0);
680685
})
681686
);
682-
let voter = run_grandpa_voter(
683-
Config {
687+
let grandpa_params = GrandpaParams {
688+
config: Config {
684689
gossip_duration: TEST_GOSSIP_DURATION,
685690
justification_period: 32,
686691
local_key,
687692
name: Some(format!("peer#{}", peer_id)),
688693
},
689-
link,
690-
MessageRouting::new(net.clone(), peer_id),
691-
InherentDataProviders::new(),
692-
Exit,
693-
).expect("all in order with client and network");
694+
link: link,
695+
network: MessageRouting::new(net.clone(), peer_id),
696+
inherent_data_providers: InherentDataProviders::new(),
697+
on_exit: Exit,
698+
telemetry_on_connect: None,
699+
};
700+
let voter = run_grandpa_voter(grandpa_params).expect("all in order with client and network");
694701

695702
runtime.spawn(voter);
696703
}
@@ -1081,18 +1088,20 @@ fn voter_persists_its_votes() {
10811088
let (_block_import, _, link) = net.lock().make_block_import(client.clone());
10821089
let link = link.lock().take().unwrap();
10831090

1084-
let mut voter = run_grandpa_voter(
1085-
Config {
1091+
let grandpa_params = GrandpaParams {
1092+
config: Config {
10861093
gossip_duration: TEST_GOSSIP_DURATION,
10871094
justification_period: 32,
10881095
local_key: Some(Arc::new(peers[0].clone().into())),
10891096
name: Some(format!("peer#{}", 0)),
10901097
},
1091-
link,
1092-
MessageRouting::new(net.clone(), 0),
1093-
InherentDataProviders::new(),
1094-
Exit,
1095-
).expect("all in order with client and network");
1098+
link: link,
1099+
network: MessageRouting::new(net.clone(), 0),
1100+
inherent_data_providers: InherentDataProviders::new(),
1101+
on_exit: Exit,
1102+
telemetry_on_connect: None,
1103+
};
1104+
let mut voter = run_grandpa_voter(grandpa_params).expect("all in order with client and network");
10961105

10971106
let voter = future::poll_fn(move || {
10981107
// we need to keep the block_import alive since it owns the

core/peerset/src/lib.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ mod slots;
2222
use std::collections::VecDeque;
2323
use futures::{prelude::*, sync::mpsc, try_ready};
2424
use libp2p::PeerId;
25+
use linked_hash_map::LinkedHashMap;
2526
use log::trace;
2627
use lru_cache::LruCache;
2728
use slots::{SlotType, SlotState, Slots};

0 commit comments

Comments
 (0)