Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: apply suggestions from code review
Co-authored-by: Jacob Heun <[email protected]>
  • Loading branch information
vasco-santos and jacobheun committed May 6, 2020
commit 1c2c4324bf6a210830cc175050b6c3f77b582cee
13 changes: 7 additions & 6 deletions doc/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -814,8 +814,6 @@ Add known `protocols` of a given peer.
peerStore.protoBook.add(peerId, protocols)
```

* [`peerStore.keyBook.get`](#peerstorekeybookget)
* [`peerStore.keyBook.set`](#peerstorekeybookset)

### peerStore.keyBook.delete

Expand All @@ -840,7 +838,7 @@ Delete the provided peer from the book.
```js
peerStore.keyBook.delete(peerId)
// false
peerStore.keyBook.set(peerId)
peerStore.keyBook.set(peerId, publicKey)
peerStore.keyBook.delete(peerId)
// true
```
Expand Down Expand Up @@ -868,7 +866,7 @@ Get the known `PublicKey` of a provided peer.
```js
peerStore.keyBook.get(peerId)
// undefined
peerStore.keyBook.set(peerId) // with inline public key
peerStore.keyBook.set(peerId, publicKey)
peerStore.keyBook.get(peerId)
// PublicKey
```
Expand All @@ -877,13 +875,14 @@ peerStore.keyBook.get(peerId)

Set known `peerId`. This can include its Public Key.

`peerStore.keyBook.set(peerId)`
`peerStore.keyBook.set(peerId, publicKey)`

#### Parameters

| Name | Type | Description |
|------|------|-------------|
| peerId | [`PeerId`][peer-id] | peerId to set |
| publicKey | [`RsaPublicKey|Ed25519PublicKey|Secp256k1PublicKey`][keys] | peer's public key |

#### Returns

Expand All @@ -894,7 +893,8 @@ Set known `peerId`. This can include its Public Key.
#### Example

```js
peerStore.keyBook.set(peerId)
const publicKey = peerId.pubKey
peerStore.keyBook.set(peerId, publicKey)
```

### peerStore.protoBook.delete
Expand Down Expand Up @@ -1420,3 +1420,4 @@ This event will be triggered anytime we are disconnected from another peer, rega
[connection]: https://github.com/libp2p/js-interfaces/tree/master/src/connection
[multiaddr]: https://github.com/multiformats/js-multiaddr
[peer-id]: https://github.com/libp2p/js-peer-id
[keys]: https://github.com/libp2p/js-libp2p-crypto/tree/master/src/keys
14 changes: 8 additions & 6 deletions src/peer-store/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ Several libp2p subsystems will perform operations, which will gather relevant in

In a libp2p node's life, it will discover peers through its discovery protocols. In a typical discovery protocol, addresses of the peer are discovered along with its peer id. Once this happens, the PeerStore should collect this information for future (or immediate) usage by other subsystems. When the information is stored, the PeerStore should inform interested parties of the peer discovered (`peer` event).

Taking into account a different scenario, a peer might perform/receive a dial request to/from a unkwown peer. In such a scenario, the PeerStore must store the peer's multiaddr once a connection is established.
Taking into account a different scenario, a peer might perform/receive a dial request to/from a unkwown peer. In such a scenario, the PeerStore must store the peer's multiaddr once a connection is established.

When a connection is being upgraded, more precisely after its encryption, or even in a discovery protocol, a libp2p node can get to know other parties public keys. In this scenario, libp2p will add the peer's public key to its `KeyBook`.
Copy link
Member Author

@vasco-santos vasco-santos May 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jacobheun from what we discussed, I could only not do this by now.

I think that it makes sense, but afaik noise uses static public keys, which I would not expect to have here. In addition, we will need to provide libp2p to the crypto module for adding the key to the keyBook or change the crypto interface to also require the public key, so that we can update this in the upgrader.js.

We should probably give some thought on this and follow up with a good solution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noise has its own keycache and I dont think we need to worry about persistence of the crypto transport keys, just the libp2p id keys. I think this section is achieved by updating the peer after we've established a connection, because the crypto handshake will result in libp2p public key exchange and verification.

Copy link
Member Author

@vasco-santos vasco-santos May 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the inbound connection, we do have the public key after the connection being established. The same is not true for the outbound connection. Both for libp2p-secio and libp2p-noise

I went deeper on secio to figure out why this is happening and I found this: https://github.com/libp2p/js-libp2p-secio/blob/master/src/handshake/crypto.js#L66-L73

When an inbound connection, we go to the else statement while in an outbound connection we go to the if side of things. In the if part we basically do not add the public key. If I just add state.id.remote = remoteId inside the if and after the validation condition, it works. Is this the expected?

In noise, I could not follow the flow so easily yet...

Copy link
Member Author

@vasco-santos vasco-santos May 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meanwhile, I already added the code for this here and a test. The validation of the outbound connection key being exchanged is currently commented as a result of what I described above

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this is fixed in secio now, I'll look into it in noise as part of my work there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you recommend here? change the test configuration to use secio for now and create an issue to track this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave it for now, I'll work on getting a patch for noise to correct this. We can add a TODO item to the peer store epic to track getting this in.


After a connection is established with a peer, the Identify protocol will run automatically. A stream is created and peers exchange their information (Multiaddrs, running protocols and their public key). Once this information is obtained, it should be added to the PeerStore. In this specific case, as we are speaking to the source of truth, we should ensure the PeerStore is prioritizing these records. If the recorded `multiaddrs` or `protocols` have changed, interested parties must be informed via the `change:multiaddrs` or `change:protocols` events respectively.

Expand Down Expand Up @@ -42,7 +44,7 @@ The `addressBook` keeps the known multiaddrs of a peer. The multiaddrs of each p

`Map<string, Address>`

A `peerId.toString()` identifier mapping to a `Address` object, which should have the following structure:
A `peerId.toB58String()` identifier mapping to a `Address` object, which should have the following structure:

```js
{
Expand All @@ -52,19 +54,19 @@ A `peerId.toString()` identifier mapping to a `Address` object, which should hav

#### Key Book

The `keyBook` tracks the publick keys of the peers by keeping their [`PeerId`][peer-id].
The `keyBook` tracks the public keys of the peers by keeping their [`PeerId`][peer-id].

`Map<string, PeerId`

A `peerId.toString()` identifier mapping to a `PeerId` of the peer. This instance contains the peer public key.
A `peerId.toB58String()` identifier mapping to a `PeerId` of the peer. This instance contains the peer public key.

#### Protocol Book

The `protoBook` holds the identifiers of the protocols supported by each peer. The protocols supported by each peer are dynamic and will change over time.

`Map<string, Set<string>>`

A `peerId.toString()` identifier mapping to a `Set` of protocol identifier strings.
A `peerId.toB58String()` identifier mapping to a `Set` of protocol identifier strings.

#### Metadata Book

Expand Down Expand Up @@ -129,4 +131,4 @@ Metadata is stored under the following key pattern:
- When improving libp2p configuration for specific runtimes, we should take into account the PeerStore recommended datastore.
- When improving libp2p configuration, we should think about a possible way of allowing the configuration of Bootstrap to be influenced by the persisted peers, as a way to decrease the load on Bootstrap nodes.

[peer-id]: https://github.com/libp2p/js-peer-id
[peer-id]: https://github.com/libp2p/js-peer-id
10 changes: 10 additions & 0 deletions src/peer-store/address-book.js
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,11 @@ class AddressBook extends Book {
this._setData(peerId, addresses)
log(`stored provided multiaddrs for ${id}`)

// Notify the existance of a new peer
if (!rec) {
this._ps.emit('peer', peerId)
}

return this
}

Expand Down Expand Up @@ -125,6 +130,11 @@ class AddressBook extends Book {

log(`added provided multiaddrs for ${id}`)

// Notify the existance of a new peer
if (!rec) {
this._ps.emit('peer', peerId)
}

return this
}

Expand Down
5 changes: 0 additions & 5 deletions src/peer-store/book.js
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,6 @@ class Book {
// Store data in memory
this.data.set(b58key, data)

// Store PeerId
if (!PeerId.isPeerId(data)) {
this._ps.keyBook.set(peerId)
}

// Emit event
emit && this._emit(peerId, data)
}
Expand Down
24 changes: 13 additions & 11 deletions src/peer-store/key-book.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ class KeyBook extends Book {
constructor (peerStore) {
super({
peerStore,
eventName: 'change:pubkey', // TODO: the name is not probably the best!?
eventName: 'change:pubkey',
eventProperty: 'pubkey',
eventTransformer: (data) => data.pubKey
})
Expand All @@ -37,12 +37,13 @@ class KeyBook extends Book {
}

/**
* Set PeerId. If the peer was not known before, it will be added.
* Set the Peer public key.
* @override
* @param {PeerId} peerId
* @param {RsaPublicKey|Ed25519PublicKey|Secp256k1PublicKey} publicKey
* @return {KeyBook}
*/
set (peerId) {
*/
set (peerId, publicKey) {
if (!PeerId.isPeerId(peerId)) {
log.error('peerId must be an instance of peer-id to store data')
throw errcode(new Error('peerId must be an instance of peer-id'), ERR_INVALID_PARAMETERS)
Expand All @@ -51,12 +52,13 @@ class KeyBook extends Book {
const id = peerId.toB58String()
const recPeerId = this.data.get(id)

!recPeerId && this._ps.emit('peer', peerId)
// If no record available, or it is incomplete
if (!recPeerId || (peerId.pubKey && !recPeerId.pubKey)) {
this._setData(peerId, peerId, {
emit: Boolean(peerId.pubKey) // No persistence if no public key
})
// If no record available, and this is valid
if (!recPeerId && publicKey) {
// This might be unecessary, but we want to store the PeerId
// to avoid an async operation when reconstructing the PeerId
peerId.pubKey = publicKey

this._setData(peerId, peerId)
log(`stored provided public key for ${id}`)
}

Expand All @@ -67,7 +69,7 @@ class KeyBook extends Book {
* Get Public key of the given PeerId, if stored.
* @override
* @param {PeerId} peerId
* @return {PublicKey}
* @return {RsaPublicKey|Ed25519PublicKey|Secp256k1PublicKey}
*/
get (peerId) {
if (!PeerId.isPeerId(peerId)) {
Expand Down
2 changes: 1 addition & 1 deletion src/peer-store/persistent/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ class PersistentPeerStore extends PeerStore {
log('create batch commit')
const batch = this._datastore.batch()
for (const peerIdStr of commitPeers) {
// PeerId (replace by keyBook)
// PeerId
const peerId = this.keyBook.data.get(peerIdStr) || PeerId.createFromB58String(peerIdStr)

// Address Book
Expand Down
32 changes: 14 additions & 18 deletions test/peer-store/key-book.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ chai.use(require('chai-bytes'))
const { expect } = chai
const sinon = require('sinon')

const PeerId = require('peer-id')
const PeerStore = require('../../src/peer-store')

const peerUtils = require('../utils/creators/peer')
Expand All @@ -24,7 +23,7 @@ describe('keyBook', () => {
kb = peerStore.keyBook
})

it('throwns invalid parameters error if invalid PeerId is provided', () => {
it('throwns invalid parameters error if invalid PeerId is provided in set', () => {
try {
kb.set('invalid peerId')
} catch (err) {
Expand All @@ -34,9 +33,19 @@ describe('keyBook', () => {
throw new Error('invalid peerId should throw error')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to change but you could instead do something like:

expect(() => kb.set('invalid peerId')).to.throw()
      .that.has.property('code', ERR_INVALID_PARAMETERS)

which avoids potentially forgetting the return/throw statements when a failure occurs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this out and I got:

AssertionError: expected [Function] to have property 'code'

I will skip this for now, we can revisit later

})

it('throwns invalid parameters error if invalid PeerId is provided in get', () => {
try {
kb.get('invalid peerId')
} catch (err) {
expect(err.code).to.equal(ERR_INVALID_PARAMETERS)
return
}
throw new Error('invalid peerId should throw error')
})

it('stores the peerId in the book and returns the public key', () => {
// Set PeerId
kb.set(peerId)
kb.set(peerId, peerId.pubKey)

// Get public key
const pubKey = kb.get(peerId)
Expand All @@ -47,22 +56,9 @@ describe('keyBook', () => {
const spy = sinon.spy(kb, '_setData')

// Set PeerId
kb.set(peerId)
kb.set(peerId)
kb.set(peerId, peerId.pubKey)
kb.set(peerId, peerId.pubKey)

expect(spy).to.have.property('callCount', 1)
})

it('stores if already stored but there was no public key stored', () => {
const spy = sinon.spy(kb, '_setData')

// Set PeerId without public key
const p = PeerId.createFromB58String(peerId.toB58String())
kb.set(p)

// Set complete peerId
kb.set(peerId)

expect(spy).to.have.property('callCount', 2)
})
})
34 changes: 34 additions & 0 deletions test/peer-store/peer-store.node.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
'use strict'
/* eslint-env mocha */

const chai = require('chai')
chai.use(require('dirty-chai'))
chai.use(require('chai-as-promised'))
const { expect } = chai
const sinon = require('sinon')

const baseOptions = require('../utils/base-options')
const peerUtils = require('../utils/creators/peer')

describe('libp2p.peerStore', () => {
let libp2p, remoteLibp2p

beforeEach(async () => {
[libp2p, remoteLibp2p] = await peerUtils.createPeer({
number: 2,
populateAddressBooks: false,
config: {
...baseOptions
}
})
})

it('adds peer address to AddressBook when establishing connection', async () => {
const spyAddressBook = sinon.spy(libp2p.peerStore.addressBook, 'add')
const remoteMultiaddr = `${remoteLibp2p.multiaddrs[0]}/p2p/${remoteLibp2p.peerId.toB58String()}`
const conn = await libp2p.dial(remoteMultiaddr)

expect(conn).to.exist()
expect(spyAddressBook).to.have.property('callCount', 1)
})
})
23 changes: 4 additions & 19 deletions test/peer-store/peer-store.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
const chai = require('chai')
chai.use(require('dirty-chai'))
const { expect } = chai
const sinon = require('sinon')

const PeerStore = require('../../src/peer-store')
const multiaddr = require('multiaddr')
Expand Down Expand Up @@ -50,25 +49,11 @@ describe('peer-store', () => {
expect(peer).to.not.exist()
})

it('sets the peer to the KeyBook when added to the AddressBook', () => {
const spyPeerStore = sinon.spy(peerStore.keyBook, 'set')
it('sets the peer\'s public key to the KeyBook', () => {
peerStore.keyBook.set(peerIds[0], peerIds[0].pubKey)

peerStore.addressBook.set(peerIds[0], [addr1, addr2])
expect(spyPeerStore).to.have.property('callCount', 1)
})

it('sets the peer to the KeyBook when added to the ProtoBook', () => {
const spyPeerStore = sinon.spy(peerStore.keyBook, 'set')

peerStore.protoBook.set(peerIds[0], [proto1])
expect(spyPeerStore).to.have.property('callCount', 1)
})

it('does not re-set the to the KeyBook when directly added to it', () => {
const spyPeerStore = sinon.spy(peerStore.keyBook, 'set')

peerStore.keyBook.set(peerIds[0])
expect(spyPeerStore).to.have.property('callCount', 1)
const pubKey = peerStore.keyBook.get(peerIds[0])
expect(pubKey).to.exist()
})
})

Expand Down
Loading