feat: initial impl of HyParView and Plumtree #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -0,0 +1,431 @@ | |||
| //! Implementation of the HyParView membership protocol | |||
* hyparview: fix forward join logic * hyparview: add a timer for timing out pending neighbor requests * hyparview: make priority an enum * plumtree: remove peers on disconnect * gossipswarm: track some stats
* prepare latency-aware test transport * add big test with N nodes (1000 default) * report some metrics and stats on the test (needs --nocapture)
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jun 27, 2023
See n0-computer/iroh-sync#1 for initial development history
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jun 28, 2023
See n0-computer/iroh-sync#1 for initial development history
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jun 29, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
2 tasks
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jun 30, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jun 30, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 6, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 7, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 12, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Member
Author
|
Closing in favor of n0-computer/iroh#1149 |
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 26, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 26, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Jul 28, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Aug 3, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
github-merge-queue bot
pushed a commit
to n0-computer/iroh
that referenced
this pull request
Aug 4, 2023
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat --------- Co-authored-by: Diva M <[email protected]>
Frando
added a commit
to n0-computer/iroh
that referenced
this pull request
Aug 4, 2023
This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find.
github-merge-queue bot
pushed a commit
to n0-computer/iroh
that referenced
this pull request
Aug 4, 2023
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
rklaehn
pushed a commit
to n0-computer/iroh-blobs
that referenced
this pull request
Oct 22, 2024
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
rklaehn
pushed a commit
to n0-computer/iroh-blobs
that referenced
this pull request
Oct 22, 2024
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
matheus23
pushed a commit
to n0-computer/iroh-doctor
that referenced
this pull request
Oct 22, 2024
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
dignifiedquire
pushed a commit
to n0-computer/iroh-gossip
that referenced
this pull request
Oct 23, 2024
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
matheus23
pushed a commit
to n0-computer/iroh
that referenced
this pull request
Nov 14, 2024
* feat: iroh-gossip initial commit This commit imports iroh-gossip into the repo. See n0-computer/iroh-sync#1 for the prehistory. * From that original PR comes my IO-less implementation of the gossip protocol (Plumtree broadcast trees and HyParView cluster membership). The implementations are quite strict according to the papers [1](https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf) and [2](https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf). If you find any deviations that are not documented, please report, this is likely a bug. * The combined protocol for Plumtree and Hyparview is additionally namespaced with a 32-byte topic ID to allow multiple broadcast trees (one per topic) to coexist. This means: Each topic is a seperate broadcast tree with seperate swarm membership and active connections (connections to the same peers are still shared on the network layer) This IO-less protocol implementation is in the `proto` module. This PR *also* adds my work from the past week: A `net` module that provides networking for the IO-less protocol impl, through the `MagicEndpoint` from iroh-net, with a (I think) quite straightforward actor implementation. The `net` module has a smoke test, and the PR also includes a `chat` example that adds signed messages on the trustless base layer. What this mostly needs is, I think, more testing and validation of the protocol implementation. For this to be practical, I think we should formulate some base usecase scenarios. The `proto` module has a very simple simulator and some tests that can take environment variables to increase the cluster size. They passed with 10.000 nodes. They also report some stats. The numbers look OK-ish, but I'd love to compare them properly with either other implementations or estimations from the original paper or other resoures. However, those are hard to find. * fix(gossip): make util pub * fix(gossip): better naming of cli arguments in chat example * chore: make clippy happy * feat: transmit opaque PeerData on join, forward and neighbor ... and wire it up in net module to transmit your endpoint addresses * example: improve gossip chat example * chore: clippy * fix(gossip): update dep version after rebase * fix: changes after rebase * change(gossip): cleanup and better debug impls * fix(gossip): disconnect handling * change: do not error on connect without addrs Before MagicEndpoint::connect would error if called without addrs or derp_region. This commit changes this by only warning and not bailing. This allows to connect to peers by peer id only if addresses or a derp_region were provided previously. The downside is that we lose a valuable error case. A more proper fix will need to move the check if a peer is dialable into the actor. * fix: adapt for changes in iroh-net, and use genawaiter * chore: clippy * docs: document public items in iroh-gossip also renames mod.rs files to modname.rs * tests: move gossip simulator into tests module * tests: make simulation tests work with top level protocol state * change: use derive_more::Debug * feat: add command to leave a topic * docs: document default plumtree config * docs: default values for hyparview * docs: add high-level description of iroh-gossip protocol * test/refactor: add unit test for plumtree optimization step * refactor: improve chat example and address review * fix: do not stop actor if conn closes before event is processed * docs: more links * fix: changes after rebase * change: remove unneeded serialize in iroh-net added earlier * fix: make tests work and cleanup some namings * chore: clippy * refactor: improve and cleanup chat example * refactor: remove unneeded InEvent variant in plumtree * refactor: improve gossip example further * docs for le people and le me * fmt * docs: plumtree * docs: hyparview * feat: validate gossip message ids on receive * chore: cleanup * feat: metrics, and backport fixes from sync-gossip-bytes branch * feat: expose which peer sent us a gossip message * fix: postcard is required * fix: only graft lazy peers in optimize_tree * chore: remove leftover comment * docs(hyparview): improve with divma's suggestions * refactor: impl IntoIterator for IndexSet * fix(hyparview): only trigger ForwardJoin if peer is not yet active * fix(hyparview): reply to shuffle if active_view.len() is 1 or 0 * refactor: rename methods for clarity * fix: do not auto-join topics on external requests * docs: typos * docs: fix * refactor: rename GossipHandle to just Gossip * refactor: less code, more derive * docx: fix * refactor: add util module for base32 parse/fmt * refactor: make PeerData optional * refactor: do not use shift_remove in indexset, not needed * test: add unittests for IndexSet * test: TimerMap and base32 utils * refactor: rename trait PeerAddress to PeerId an generic PA to PI * feat: evict messages after some time * chore: clippy & fmt * fix: renames in doc comments * fix: use rand_chacha in tests for predictable cross-arch results * chore: fmt * chore: remove unneeded line * test: try to fix cross-arch rngs * test: always recreate RNG for cross-arch compat * fix: change after rebase on main * chore: remove debug leftovers --------- Co-authored-by: Diva M <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the initial implementation of a
gossipswarmmodule. The module exposes a protocol that combines HyParView and Plumtree.The implementation is IO-less. It exposes a single public method
handlewhere events are passed in (received messages, expired timers, application commands) and an iterator of events to process is returned (messages to send, timers to be scheduled, application events).The implementation also is generic over a
PeerAddress, which is an identifier chosen for each peer by the upper protocol/networking level (not yet implemented).An initial smoke test for both HyParView and Plumtree passes. More tests are needed of course. Notably, the passive view management, lazy pushes andgossiping over time (basically all timers) are not yet tested.