Request based availability distribution #2423

eskimor · 2021-02-11T17:29:07Z

PR went large, because I took the occasion to make ValidatorIndex a newtype wrapper, which is quite invasive in test code.

What it does in a nutshell:

It monitors occupied cores and keeps track of them.
For each occupied core that does not belong to our own backing group, we spawn a task for fetching/validating/storing the chunk. For fetching, we try each validator of the backing group in a random order, that stays fixed over the session. One exception: If validators did not deliver, they will be ranked back, so we won't waste time at each block, waiting for unresponsive validators. Maintaining the order is important, so networking will keep TCP connections open, when querying the same validator each block. We also save resources, by not opening more connections than necessary.
For each requested chunk: Fetch it from the store and deliver it.

Status: Tested manually and seems to work fine (adder-collator).

Still missing in this PR:

Tests
Instrumentation/metrics/jaeger
Guide update

Work on the requesting side of things.

as I will likely replace it with something smarter.

and getting things to type check.

primitives/src/v0.rs

- Compile fixes. - Cleanup.

Missing: - Check for cancel - Actual querying of peer ids.

- also make it impossible to check the wrong chunk index. - Export needed function in validator_discovery.

Also make sure to not fetch our own chunk.

- FetchTask::is_finished - SessionCache::fetch_session_info

adder-collator is running!

rphmeier · 2021-02-24T00:13:10Z

I filed follow-up issues for the two points I raised: K ancestors and parallelization. This PR just needs tests and then we're good to go.

eskimor · 2021-02-24T07:27:58Z

I filed follow-up issues for the two points I raised: K ancestors and parallelization. This PR just needs tests and then we're good to go.

uh perfect, then tests now :-)

node/network/availability-distribution/src/metrics.rs

rphmeier · 2021-02-26T17:56:07Z

node/network/bridge/src/lib.rs

+		assert_matches!(
+			virtual_overseer.recv().await,
+			AllMessages::AvailabilityRecovery(
+				AvailabilityRecoveryMessage::NetworkBridgeUpdateV1(e)


This can't be triggered by a malicious node, can it?

(oh, this is test code 👍 )

dvdplm · 2021-03-03T08:31:05Z

* Instrumentation/metrics/jaeger

@eskimor Is there an issue open for adding back the jaeger spans that we can track?

eskimor · 2021-03-03T10:18:38Z

Good catch - no, it is just on my list. Will make an issue: #2551

eskimor added 5 commits February 4, 2021 10:05

WIP

6f2e0e9

availability distribution, still very wip.

da850f7

Work on the requesting side of things.

Some docs on what I intend to do.

fbf0ec1

Checkpoint of session cache implementation

ac543c1

as I will likely replace it with something smarter.

More work, mostly on cache

07f6bc3

and getting things to type check.

github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Feb 11, 2021

ordian reviewed Feb 11, 2021

View reviewed changes

primitives/src/v0.rs Outdated Show resolved Hide resolved

eskimor added 7 commits February 12, 2021 10:51

Only derive MallocSizeOf and Debug for std.

ef84ea5

availability-distribution: Cache feature complete.

1e35804

Sketch out logic in FetchTask for actual fetching.

d8fda81

- Compile fixes. - Cleanup.

Format cleanup.

47036c9

More format fixes.

4ad902f

Almost feature complete fetch_task.

fee9476

Missing: - Check for cancel - Actual querying of peer ids.

Finish FetchTask so far.

b9aa906

eskimor mentioned this pull request Feb 16, 2021

Optimize Availability Distribution: direct distribution #1979

Closed

eskimor added 15 commits February 16, 2021 16:01

Directly use AuthorityDiscoveryId in protocol and cache.

a65562f

Resolve AuthorityDiscoveryId on sending requests.

4a43561

Rework fetch_task

6543b30

- also make it impossible to check the wrong chunk index. - Export needed function in validator_discovery.

From<u32> implementation for ValidatorIndex.

256e559

Fixes and more integration work.

f8d5fef

Make session cache proper lru cache.

5e77fb4

Use proper lru cache.

72704ee

Requester finished.

60a2faf

ProtocolState -> Requester

452b55f

Also make sure to not fetch our own chunk.

Cleanup + fixes.

2b9b983

Remove unused functions

d683f10

- FetchTask::is_finished - SessionCache::fetch_session_info

availability-distribution responding side.

d7a8a31

Cleanup + Fixes.

3fed607

More fixes.

39d6bc2

More fixes.

49b1764

adder-collator is running!

eskimor added 2 commits February 23, 2021 19:38

Fix indentation.

4d05d00

Use typed default timeout constant.

aadc80f

eskimor mentioned this pull request Feb 23, 2021

Request availability chunks from actual backers paritytech/polkadot-sdk#976

Open

eskimor added 5 commits February 23, 2021 20:06

Make channel size 0, as each sender gets one slot anyways.

e45f61c

Fix incorrect metrics initialization.

43dfd1c

Merge branch 'master' into rk-availability-distribution-2306

e262782

Fix build after merge.

5353157

More fixes.

ff94444

This was referenced Feb 24, 2021

Parallelize Requesting of Availability Chunks in Distribution #2512

Open

Availability Distribution: Consider occupied cores from last k blocks within same session #2513

Closed

Hopefully valid metrics names.

6b71e54

ordian reviewed Feb 25, 2021

View reviewed changes

node/network/availability-distribution/src/metrics.rs Outdated Show resolved Hide resolved

eskimor added 6 commits February 25, 2021 11:04

Better metrics names.

190adaa

Some tests that already work.

8901344

Slightly better docs.

1d29b5c

Some more tests.

83ff666

Merge branch 'master' into rk-availability-distribution-2306

e9210e5

Fix network bridge test.

a0e01ec

rphmeier reviewed Feb 26, 2021

View reviewed changes

rphmeier approved these changes Feb 26, 2021

View reviewed changes

rphmeier merged commit 950447e into master Feb 26, 2021

rphmeier deleted the rk-availability-distribution-2306 branch February 26, 2021 17:58

eskimor mentioned this pull request Mar 3, 2021

Add back Jaeger spans to availability distribution #2551

Closed

ordian mentioned this pull request Mar 3, 2021

Use Request/Response protocol for availability distribution #2306

Closed

slumber mentioned this pull request Dec 23, 2021

availability-distribution: look for leaf ancestors within the same session #4596

Merged

1 task

eskimor mentioned this pull request Nov 11, 2022

Member request polkadot-fellows/seeding#57

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Request based availability distribution #2423

Request based availability distribution #2423

Uh oh!

eskimor commented Feb 11, 2021 •

edited

Loading

Uh oh!

Uh oh!

rphmeier commented Feb 24, 2021

Uh oh!

eskimor commented Feb 24, 2021

Uh oh!

Uh oh!

rphmeier Feb 26, 2021

Uh oh!

rphmeier Feb 26, 2021

Uh oh!

dvdplm commented Mar 3, 2021

Uh oh!

eskimor commented Mar 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Request based availability distribution #2423

Request based availability distribution #2423

Uh oh!

Conversation

eskimor commented Feb 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

rphmeier commented Feb 24, 2021

Uh oh!

eskimor commented Feb 24, 2021

Uh oh!

Uh oh!

rphmeier Feb 26, 2021

Choose a reason for hiding this comment

Uh oh!

rphmeier Feb 26, 2021

Choose a reason for hiding this comment

Uh oh!

dvdplm commented Mar 3, 2021

Uh oh!

eskimor commented Mar 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

eskimor commented Feb 11, 2021 •

edited

Loading