Peerset::discovered accepts many peer ids #2213

debris · 2019-04-04T19:07:07Z

reduces number of calls to alloc_slots which always pops a peer from discovered list

debris · 2019-04-04T19:07:59Z

core/network-libp2p/src/custom_proto/behaviour.rs


 	/// Indicates to the peerset that we have discovered new addresses for a given node.
-	pub fn add_discovered_node(&mut self, peer_id: &PeerId) {
-		debug!(target: "sub-libp2p", "PSM <= Discovered({:?})", peer_id);


not sure what to do about this debug!. I could move print it in Peerset::discovered if needed

I think it would be nice to have the logging both in sub-libp2p and in the peerset.
I guess we could have a custom iterator that does the same as Iterator::map but also implements Drop and prints the elements that are ignored?

Thanks for suggestion. I realised that we can just print all items inside .map combinator. I fixed it in the latest commit

tomaka · 2019-04-04T19:27:36Z

core/network-libp2p/src/custom_proto/behaviour.rs


 	/// Indicates to the peerset that we have discovered new addresses for a given node.
-	pub fn add_discovered_node(&mut self, peer_id: &PeerId) {
-		debug!(target: "sub-libp2p", "PSM <= Discovered({:?})", peer_id);


I think it would be nice to have the logging both in sub-libp2p and in the peerset.
I guess we could have a custom iterator that does the same as Iterator::map but also implements Drop and prints the elements that are ignored?

debris · 2019-04-04T21:24:08Z

core/peerset/src/lib.rs

-			return;
+	pub fn discovered<I: IntoIterator<Item = PeerId>>(&mut self, peer_ids: I) {
+		for peer_id in peer_ids {
+			if !self.data.in_slots.contains(&peer_id) && !self.data.out_slots.contains(&peer_id) && !self.data.discovered.contains(&peer_id) {


I added this additional check in data.discovered, so we can print peer_id without cloning it

* Peerset::discovered accepts many peer ids * Improve tracing in peerset

…y-by-default' Squashed commit of the following: commit 6de583a Author: Xiliang Chen <[email protected]> Date: Wed Apr 10 20:26:29 2019 +1200 update authers for rest of the node-template cargo.toml files (#2242) commit 5240bc1 Author: Bastian Köcher <[email protected]> Date: Tue Apr 9 10:31:18 2019 +0200 Throw a compile error for `on_finalise` and `on_initialise` (#2236) commit 67d2e71 Author: Pierre Krieger <[email protected]> Date: Tue Apr 9 05:30:43 2019 -0300 Add warning when using default protocol ID (#2234) * Add warning when using default protocol ID * Update core/service/src/lib.rs commit 1421fed Author: Xiliang Chen <[email protected]> Date: Tue Apr 9 17:22:20 2019 +1200 update name and authors to placeholder text for node-template (#2222) * update name and authors to placeholder text * revert package name change commit 6617f23 Author: André Silva <[email protected]> Date: Mon Apr 8 12:50:34 2019 +0100 grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (#2139) * core: grandpa: migrate to grandpa 0.7 * core: grandpa: store current round votes and load them on startup * core: grandpa: resend old persisted votes for the current round * core: grandpa: store base and votes for last completed round * core: grandpa: fix latest grandpa 0.7 changes * core: grandpa: update to grandpa 0.7.1 * core: grandpa: persist votes for last two completed rounds * core: grandpa: simplify VoterSetState usage * core: grandpa: use Environment::update_voter_set_state * core: grandpa: fix aux_schema test * core: grandpa: add docs * core: grandpa: add note about environment assumption * core: grandpa: don't update voter set state on ignored votes * core: grandpa: add test for v1 -> v2 aux_schema migration * core: grandpa: add test for voter vote persistence * core: grandpa: use grandpa 0.7.1 from crates.io * core: grandpa: use try_init in test * core: grandpa: add comment about block_import in test * core: grandpa: avoid cloning HasVoted * core: grandpa: add missing docs * core: grandpa: cleanup up can_propose/prevote/precommit commit 21e0877 Author: Gregory Terzian <[email protected]> Date: Mon Apr 8 13:17:00 2019 +0200 remove clone bound on specialization in testnet factory (#2157) commit 7c64746 Author: Andrew Jones <[email protected]> Date: Sat Apr 6 12:23:56 2019 +0100 Contract import/export validation (#2203) * Reject validation of contract with unknown exports * Validate imports eagerly * Increment spec version commit 12718fa Author: Pierre Krieger <[email protected]> Date: Fri Apr 5 14:07:09 2019 -0300 Fix state inconsistency between handler and behaviour (#2220) * Fix state inconsistency between handler and behaviour * Fix the error! being in the wrong place commit f917d12 Author: Bastian Köcher <[email protected]> Date: Fri Apr 5 18:50:38 2019 +0200 Use `storage_root` of newly calculated header (#2216) Instead of calculating the `storage_root` a second time, we just can take the `storage_root` from the new header. commit 3359ce0 Author: Marek Kotewicz <[email protected]> Date: Fri Apr 5 14:44:46 2019 +0200 Peerset::discovered accepts many peer ids (#2213) * Peerset::discovered accepts many peer ids * Improve tracing in peerset commit dd82e0e Author: Marek Kotewicz <[email protected]> Date: Thu Apr 4 19:40:40 2019 +0200 simplification of peerset api (#2123) * Introduction of PeersetHandle * integrate PeersetHandle with the rest of the codebase * fix compilation errors * more tests for peerset, fixed overwriting bug in add_reserved_peer * Slots data structure and bugfixes for peerset * bend to pressure * updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2 * peerset discovered list is now a LinkedHashMap * fix review suggestions * split back Peerset and PeersetHandle * test for Peerset::discovered * applied review suggestions * fixes to peerset::incoming * peerset disconnects are all instantaneous * instantaneous drop in peerset finished * Peerset::set_reserved_only can also reconnect nodes * Peerset scores cache uses lru-cache * remove redundant function call and comment from Peerset::on_set_reserved_only * add_peer returns SlotState enum * apply review suggestions * is_reserved -> is_connected_and_reserved commit d90833d Author: Arkadiy Paronyan <[email protected]> Date: Thu Apr 4 18:01:28 2019 +0200 Disconnect on protocol timeout (#2212) commit c0a46b5 Author: André Silva <[email protected]> Date: Thu Apr 4 15:56:49 2019 +0100 core: grandpa: verify commit target in justification (#2201) commit 3a4901a Author: Bastian Köcher <[email protected]> Date: Thu Apr 4 16:56:16 2019 +0200 Introduce `original_storage` and `original_storage_hash` (#2211) Both functions will ignore any overlayed changes and access the backend directly. commit a7a469f Author: Xiliang Chen <[email protected]> Date: Fri Apr 5 03:55:55 2019 +1300 code cleanup (#2206) commit 26c7b44 Author: Arkadiy Paronyan <[email protected]> Date: Wed Apr 3 15:52:46 2019 +0200 Emberic elm testnet (#2197)

* Peerset::discovered accepts many peer ids * Improve tracing in peerset

* Send high-level consensus telemetry by default * Notify telemetry on finalized * Send used authority set to telemetry * Do not send commit message telemetry by default * Fix typo * Allow for notifications on telemetry connect ...and send the current authority set on each connect. * Send authority set to telemetry on change * Revert "Send used authority set to telemetry" This reverts commit 1deceea. * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' Squashed commit of the following: commit 6de583a Author: Xiliang Chen <[email protected]> Date: Wed Apr 10 20:26:29 2019 +1200 update authers for rest of the node-template cargo.toml files (#2242) commit 5240bc1 Author: Bastian Köcher <[email protected]> Date: Tue Apr 9 10:31:18 2019 +0200 Throw a compile error for `on_finalise` and `on_initialise` (#2236) commit 67d2e71 Author: Pierre Krieger <[email protected]> Date: Tue Apr 9 05:30:43 2019 -0300 Add warning when using default protocol ID (#2234) * Add warning when using default protocol ID * Update core/service/src/lib.rs commit 1421fed Author: Xiliang Chen <[email protected]> Date: Tue Apr 9 17:22:20 2019 +1200 update name and authors to placeholder text for node-template (#2222) * update name and authors to placeholder text * revert package name change commit 6617f23 Author: André Silva <[email protected]> Date: Mon Apr 8 12:50:34 2019 +0100 grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (#2139) * core: grandpa: migrate to grandpa 0.7 * core: grandpa: store current round votes and load them on startup * core: grandpa: resend old persisted votes for the current round * core: grandpa: store base and votes for last completed round * core: grandpa: fix latest grandpa 0.7 changes * core: grandpa: update to grandpa 0.7.1 * core: grandpa: persist votes for last two completed rounds * core: grandpa: simplify VoterSetState usage * core: grandpa: use Environment::update_voter_set_state * core: grandpa: fix aux_schema test * core: grandpa: add docs * core: grandpa: add note about environment assumption * core: grandpa: don't update voter set state on ignored votes * core: grandpa: add test for v1 -> v2 aux_schema migration * core: grandpa: add test for voter vote persistence * core: grandpa: use grandpa 0.7.1 from crates.io * core: grandpa: use try_init in test * core: grandpa: add comment about block_import in test * core: grandpa: avoid cloning HasVoted * core: grandpa: add missing docs * core: grandpa: cleanup up can_propose/prevote/precommit commit 21e0877 Author: Gregory Terzian <[email protected]> Date: Mon Apr 8 13:17:00 2019 +0200 remove clone bound on specialization in testnet factory (#2157) commit 7c64746 Author: Andrew Jones <[email protected]> Date: Sat Apr 6 12:23:56 2019 +0100 Contract import/export validation (#2203) * Reject validation of contract with unknown exports * Validate imports eagerly * Increment spec version commit 12718fa Author: Pierre Krieger <[email protected]> Date: Fri Apr 5 14:07:09 2019 -0300 Fix state inconsistency between handler and behaviour (#2220) * Fix state inconsistency between handler and behaviour * Fix the error! being in the wrong place commit f917d12 Author: Bastian Köcher <[email protected]> Date: Fri Apr 5 18:50:38 2019 +0200 Use `storage_root` of newly calculated header (#2216) Instead of calculating the `storage_root` a second time, we just can take the `storage_root` from the new header. commit 3359ce0 Author: Marek Kotewicz <[email protected]> Date: Fri Apr 5 14:44:46 2019 +0200 Peerset::discovered accepts many peer ids (#2213) * Peerset::discovered accepts many peer ids * Improve tracing in peerset commit dd82e0e Author: Marek Kotewicz <[email protected]> Date: Thu Apr 4 19:40:40 2019 +0200 simplification of peerset api (#2123) * Introduction of PeersetHandle * integrate PeersetHandle with the rest of the codebase * fix compilation errors * more tests for peerset, fixed overwriting bug in add_reserved_peer * Slots data structure and bugfixes for peerset * bend to pressure * updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2 * peerset discovered list is now a LinkedHashMap * fix review suggestions * split back Peerset and PeersetHandle * test for Peerset::discovered * applied review suggestions * fixes to peerset::incoming * peerset disconnects are all instantaneous * instantaneous drop in peerset finished * Peerset::set_reserved_only can also reconnect nodes * Peerset scores cache uses lru-cache * remove redundant function call and comment from Peerset::on_set_reserved_only * add_peer returns SlotState enum * apply review suggestions * is_reserved -> is_connected_and_reserved commit d90833d Author: Arkadiy Paronyan <[email protected]> Date: Thu Apr 4 18:01:28 2019 +0200 Disconnect on protocol timeout (#2212) commit c0a46b5 Author: André Silva <[email protected]> Date: Thu Apr 4 15:56:49 2019 +0100 core: grandpa: verify commit target in justification (#2201) commit 3a4901a Author: Bastian Köcher <[email protected]> Date: Thu Apr 4 16:56:16 2019 +0200 Introduce `original_storage` and `original_storage_hash` (#2211) Both functions will ignore any overlayed changes and access the backend directly. commit a7a469f Author: Xiliang Chen <[email protected]> Date: Fri Apr 5 03:55:55 2019 +1300 code cleanup (#2206) commit 26c7b44 Author: Arkadiy Paronyan <[email protected]> Date: Wed Apr 3 15:52:46 2019 +0200 Emberic elm testnet (#2197) * Make telemetry onconnect hoook optional * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' * Introduce GrandpaParams struct to condense parameters * Remove debug statement * Fix tests * Rename parameter * Fix tests * Rename struct * Do not send verbosity level * Combine imports * Implement comments * Run cargo build --all * Remove noisy telemetry * Add docs for public items * Unbox and support Clone trait * Fix merge * Fix merge * Update core/finality-grandpa/src/lib.rs Co-Authored-By: cmichi <[email protected]>

* Send high-level consensus telemetry by default * Notify telemetry on finalized * Send used authority set to telemetry * Do not send commit message telemetry by default * Fix typo * Allow for notifications on telemetry connect ...and send the current authority set on each connect. * Send authority set to telemetry on change * Revert "Send used authority set to telemetry" This reverts commit 1deceea. * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' Squashed commit of the following: commit 6de583a Author: Xiliang Chen <[email protected]> Date: Wed Apr 10 20:26:29 2019 +1200 update authers for rest of the node-template cargo.toml files (paritytech#2242) commit 5240bc1 Author: Bastian Köcher <[email protected]> Date: Tue Apr 9 10:31:18 2019 +0200 Throw a compile error for `on_finalise` and `on_initialise` (paritytech#2236) commit 67d2e71 Author: Pierre Krieger <[email protected]> Date: Tue Apr 9 05:30:43 2019 -0300 Add warning when using default protocol ID (paritytech#2234) * Add warning when using default protocol ID * Update core/service/src/lib.rs commit 1421fed Author: Xiliang Chen <[email protected]> Date: Tue Apr 9 17:22:20 2019 +1200 update name and authors to placeholder text for node-template (paritytech#2222) * update name and authors to placeholder text * revert package name change commit 6617f23 Author: André Silva <[email protected]> Date: Mon Apr 8 12:50:34 2019 +0100 grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (paritytech#2139) * core: grandpa: migrate to grandpa 0.7 * core: grandpa: store current round votes and load them on startup * core: grandpa: resend old persisted votes for the current round * core: grandpa: store base and votes for last completed round * core: grandpa: fix latest grandpa 0.7 changes * core: grandpa: update to grandpa 0.7.1 * core: grandpa: persist votes for last two completed rounds * core: grandpa: simplify VoterSetState usage * core: grandpa: use Environment::update_voter_set_state * core: grandpa: fix aux_schema test * core: grandpa: add docs * core: grandpa: add note about environment assumption * core: grandpa: don't update voter set state on ignored votes * core: grandpa: add test for v1 -> v2 aux_schema migration * core: grandpa: add test for voter vote persistence * core: grandpa: use grandpa 0.7.1 from crates.io * core: grandpa: use try_init in test * core: grandpa: add comment about block_import in test * core: grandpa: avoid cloning HasVoted * core: grandpa: add missing docs * core: grandpa: cleanup up can_propose/prevote/precommit commit 21e0877 Author: Gregory Terzian <[email protected]> Date: Mon Apr 8 13:17:00 2019 +0200 remove clone bound on specialization in testnet factory (paritytech#2157) commit 7c64746 Author: Andrew Jones <[email protected]> Date: Sat Apr 6 12:23:56 2019 +0100 Contract import/export validation (paritytech#2203) * Reject validation of contract with unknown exports * Validate imports eagerly * Increment spec version commit 12718fa Author: Pierre Krieger <[email protected]> Date: Fri Apr 5 14:07:09 2019 -0300 Fix state inconsistency between handler and behaviour (paritytech#2220) * Fix state inconsistency between handler and behaviour * Fix the error! being in the wrong place commit f917d12 Author: Bastian Köcher <[email protected]> Date: Fri Apr 5 18:50:38 2019 +0200 Use `storage_root` of newly calculated header (paritytech#2216) Instead of calculating the `storage_root` a second time, we just can take the `storage_root` from the new header. commit 3359ce0 Author: Marek Kotewicz <[email protected]> Date: Fri Apr 5 14:44:46 2019 +0200 Peerset::discovered accepts many peer ids (paritytech#2213) * Peerset::discovered accepts many peer ids * Improve tracing in peerset commit dd82e0e Author: Marek Kotewicz <[email protected]> Date: Thu Apr 4 19:40:40 2019 +0200 simplification of peerset api (paritytech#2123) * Introduction of PeersetHandle * integrate PeersetHandle with the rest of the codebase * fix compilation errors * more tests for peerset, fixed overwriting bug in add_reserved_peer * Slots data structure and bugfixes for peerset * bend to pressure * updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2 * peerset discovered list is now a LinkedHashMap * fix review suggestions * split back Peerset and PeersetHandle * test for Peerset::discovered * applied review suggestions * fixes to peerset::incoming * peerset disconnects are all instantaneous * instantaneous drop in peerset finished * Peerset::set_reserved_only can also reconnect nodes * Peerset scores cache uses lru-cache * remove redundant function call and comment from Peerset::on_set_reserved_only * add_peer returns SlotState enum * apply review suggestions * is_reserved -> is_connected_and_reserved commit d90833d Author: Arkadiy Paronyan <[email protected]> Date: Thu Apr 4 18:01:28 2019 +0200 Disconnect on protocol timeout (paritytech#2212) commit c0a46b5 Author: André Silva <[email protected]> Date: Thu Apr 4 15:56:49 2019 +0100 core: grandpa: verify commit target in justification (paritytech#2201) commit 3a4901a Author: Bastian Köcher <[email protected]> Date: Thu Apr 4 16:56:16 2019 +0200 Introduce `original_storage` and `original_storage_hash` (paritytech#2211) Both functions will ignore any overlayed changes and access the backend directly. commit a7a469f Author: Xiliang Chen <[email protected]> Date: Fri Apr 5 03:55:55 2019 +1300 code cleanup (paritytech#2206) commit 26c7b44 Author: Arkadiy Paronyan <[email protected]> Date: Wed Apr 3 15:52:46 2019 +0200 Emberic elm testnet (paritytech#2197) * Make telemetry onconnect hoook optional * Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default' * Introduce GrandpaParams struct to condense parameters * Remove debug statement * Fix tests * Rename parameter * Fix tests * Rename struct * Do not send verbosity level * Combine imports * Implement comments * Run cargo build --all * Remove noisy telemetry * Add docs for public items * Unbox and support Clone trait * Fix merge * Fix merge * Update core/finality-grandpa/src/lib.rs Co-Authored-By: cmichi <[email protected]>

Peerset::discovered accepts many peer ids

35be41a

debris added A0-please_review Pull request needs code review. M4-core labels Apr 4, 2019

debris commented Apr 4, 2019

View reviewed changes

debris requested a review from tomaka April 4, 2019 19:08

tomaka suggested changes Apr 4, 2019

View reviewed changes

tomaka added A6-mustntgrumble and removed A0-please_review Pull request needs code review. labels Apr 4, 2019

Improve tracing in peerset

e920d3b

debris force-pushed the peerset-discovered branch from 513f150 to e920d3b Compare April 4, 2019 21:20

debris commented Apr 4, 2019

View reviewed changes

tomaka approved these changes Apr 5, 2019

View reviewed changes

bkchr merged commit 3359ce0 into master Apr 5, 2019

bkchr deleted the peerset-discovered branch April 5, 2019 12:44

cmichi pushed a commit that referenced this pull request Apr 10, 2019

Peerset::discovered accepts many peer ids (#2213)

f280235

* Peerset::discovered accepts many peer ids * Improve tracing in peerset

cmichi pushed a commit that referenced this pull request Apr 10, 2019

Peerset::discovered accepts many peer ids (#2213)

23537e0

* Peerset::discovered accepts many peer ids * Improve tracing in peerset

MTDK1 pushed a commit to bdevux/substrate that referenced this pull request Apr 12, 2019

Peerset::discovered accepts many peer ids (paritytech#2213)

5a10574

* Peerset::discovered accepts many peer ids * Improve tracing in peerset

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Peerset::discovered accepts many peer ids #2213

Peerset::discovered accepts many peer ids #2213

Uh oh!

debris commented Apr 4, 2019

Uh oh!

debris Apr 4, 2019

Uh oh!

tomaka Apr 4, 2019

Uh oh!

debris Apr 4, 2019

Uh oh!

tomaka Apr 4, 2019

Uh oh!

debris Apr 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Peerset::discovered accepts many peer ids #2213

Peerset::discovered accepts many peer ids #2213

Uh oh!

Conversation

debris commented Apr 4, 2019

Uh oh!

debris Apr 4, 2019

Choose a reason for hiding this comment

Uh oh!

tomaka Apr 4, 2019

Choose a reason for hiding this comment

Uh oh!

debris Apr 4, 2019

Choose a reason for hiding this comment

Uh oh!

tomaka Apr 4, 2019

Choose a reason for hiding this comment

Uh oh!

debris Apr 4, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants