Shared Key DM

[arthurfranca]

This is an alternative to #686 Sealed Gift-wrapped DMs (SGWDMs).

The SGWDMs hides metadata but:

1. The receiver has no way to fetch just DMs (Gift-wrap is a generic kind:1059 event)
2. The receiver has no way to fetch just Gift-wraps/DMs from a specific person
3. The receiver needs to fetch messages from the "last read at" moment minus one week onwards

This all translates into the receiver fetching too much events it doesn't want;

Read here

[vitorpamplona]

Feels like a lot of work to get a "chatroom-id" design. If you just want to make a message queue between two people, you could get the conversation key from NIP-44 and turn that into a nostr private key. Both parties then download everything from the pubkey of that conversation key and they also encrypt their messages to that address. Then you don't need to negotiate a nym between the two parties. It becomes a slightly better NIP-04. One of the group proposals was like that, but using NIP-04 encryption.

You can use this NYM design with #686. Since the client will have a list of Nyms to keep an eye on, it could implement #686 but use the NYM pubkey as the receiver of the GiftWrap. Since only DMs are using that Nym, you have successfully filtered all DMs from all GiftWraps (solved your point 1 and 2) without revealing they are DMs to anyone else. Implementers of the NYM spec can then send the GiftWraps with the correct dates to solve for 3, while integrating with everyone else doing #686

[arthurfranca]

Using conversation key as privkey may be a good idea but what if getConversationKey ends up not being available on nip44 v3?

Either way, if checking for new messages, I don't want to have to check for { "authors": [<my 3k follows>] }. Also, what about non-follows? So even if using conversation key as privkey, there would need to exist at least a "chat request" event, used as a way to choose the subset of nyms to use as authors filter field.

Implementers of the NYM spec can then send the GiftWraps with the correct dates to solve for 3, while integrating with everyone else doing #686

We probably should choose one winning spec / merge the best features of both. I don't see how some clients using correct dates and others back-dates would be compatible.

[arthurfranca]

I don't know how other people feel about a client going back 1 week and fetching many duplicate gift wraps it already had using #686. I wonder if spam would also be a problem when fetching gift wraps sent to a specific user even if its just current time onwards.

Maybe letting spam be limited to "chat requests" (like if ["p", "<receiver-pubkey>"] wouldn't be present on the gift wrap, but just on "chat request" events) would be better 🤔️?

Maybe I'm just overthinking and #686 is alright xD

[vitorpamplona]

Using conversation key as privkey may be a good idea but what if getConversationKey ends up #940 (comment)?

If there is a good reason to exist, the method will be there.

I don't want to have to check for { "authors": [<my 3k follows>] }

Ohhh, so you want to reuse the same nyms for many contacts? Otherwise, the nym list will likely be bigger than the follow list. That would break the solution for 2, though.

We probably should choose one winning spec / merge the best features of both.

I don't think there will be just one spec for very different use cases / privacy needs. But we can try.

I don't see how some clients using correct dates and others back-dates would be compatible.

It would be because you know which Wraps are for nyms and thus do not use random dates. Chat UI is all the same. The nym would just add a faster way to download the DMs from the people you care about. But clients will always have to check the GiftWraps as well. I think Aliases will have to exist at some point: #716 (comment)

I don't know how other people feel about a client going back 1 week and fetching many duplicate gift wraps it already had using #686.

This was designed with the SYNC capabilities in mind: #826

Once a sync implementation becomes the norm (it will have to, sooner or later), it will be easier to avoid redownloads.

But, as of now, re-downloads don't seem to be a big issue. Even for heavy DM users like myself.

Maybe I'm just overthinking and #686 is alright xD

I always recommend implementing it before making these analyses. That spec has hidden complications (like database and caching management), but it is quite reusable for other event kinds once clients have it set up.

[arthurfranca]

If there is a good reason to exist, the method will be there.

true

A: I don't want to have to check for { "authors": [<my 3k follows>] }
V: Ohhh, so you want to reuse the same nyms for many contacts? Otherwise, the nym list will likely be bigger than the follow list. That would break the solution for 2, though.

I meant an user most likely won't talk to most of its follows cause most of them aren't their "friends". So a "Nym List" filled by accepting chat-request-like events isn't supposed to include that many entries. On the other hand, simply using the follow list to track conversation-key-based-nyms would include all hypothetical 3k follows and also leave out non-follows.

But, as of now, re-downloads don't seem to be a big issue. Even for heavy DM users like myself.

It could be a blow to the user mobile data plan though. Receiving spam at ["p", <user>] may use precious MBs too.

[vitorpamplona]

It could be a blow to the user mobile data plan though. Receiving spam at ["p", ] may use precious MBs too.

That can happen with any event kind the client supports. NIP-04s, kind 1s, reactions, etc. Even these NYMs, since they are public and can be seen by anyone, can be the target of spam.

[arthurfranca]

@vitorpamplona Pivoted to your idea of using the conversation key. What do you think?

[vitorpamplona]

Much simpler. I am not sure what the 30078 event is doing. Maybe just replace it with one of the Auth events?

[arthurfranca]

The 30078 is the NIP-78 "Anything You Want" event just to prove the user owns the pubkey. Changed it to a specific kind with some added characteristics.

[vitorpamplona]

I wonder if it is better to keep things consistent and use random keys to sign but p-tag the conversation key...

In that way, it's impossible for the public to know if this is somebody's "nym"/shared key or not.

[arthurfranca]

The p tag downside is someone can flood the 2 participants with bogus gift wraps, forcing them to download garbage just for fun.

But you have a good point.

While we don't decide the best approach I will just add a random p tag instead to make it look like a regular gift wrap. Of course this is rather weak as repeating the pubkey field value on gift wraps is supposed to be very hard to happen.

[vitorpamplona]

The p tag downside is someone can flood the 2 participants with bogus gift wraps, forcing them to download garbage just for fun.

They can do the same with the pubkey... With or without the random p tag. They just need to see many wraps shipped to the same pubkey and blast it.

[arthurfranca]

Not in the case of DMs cause the filter to fetch them will be { authors: ["<conversation-key-AB-derived-pubkey>"], kinds: [1059] } and only the participants can sign valid events with that key (relays aren't supposed to store events with wrong sig).

While if using { #p: ["<conversation-key-AB-derived-pubkey>"], kinds: [1059] } anyone can send a valid event with that p tag.

[vitorpamplona]

oh yes, makes sense.

[arthurfranca]

Yes "Chat Requests" can be spammed though I think it's a lesser problem to the user cause 1) the spam won't pollute the active chats list; 2) it may not be that tasty of a target to spammers cause it won't carry a message with links and such; 3) The client may not need to download all chat requests (read below).

With number 3 I mean the receiver client could just fetch the last N chat requests and ignore the rest. N could be a number relative to the number of days the user hasn't opened the app or simply a fixed number like 500. I wouldn't store/sync chat requests on the local DB at all, just always fetch a recent list from relays.
This works ok if the sender client is wise enough to send duplicate chat requests to the same user that has never replied with a DM just in case the receiver missed the previous request.

This NIP would force the client to have one tab for approving chat requests (may be flooded yeah) and another tab for all (approved) DMs with no spam.

[arthurfranca]

This works ok if the sender client is wise enough to send duplicate chat requests...

I tried to highlight this a bit when I wrote on the NIP this: "If user A has sent a considerable amount of messages with no reply from user B, it can send to B another "Chat Request" event."

[vitorpamplona]

This NIP would force the client to have one tab for approving chat requests (may be flooded yeah) and another tab for all (approved) DMs with no spam.

Which is the same we do for regular GiftWrapped DMs.

My point is that if you are trying to solve the spam bloat, this is not doing it. The client still needs to download all requests (which can be made huge -- 100KB each) to see which ones are real and which ones are not.

[arthurfranca]

Yes someone can flood your chat request inbox and unfortunately this can't be avoided but just rate limited by relays =(

There's a slight difference though on regular non-malicious usage: An user A is supposed to send one or a few chat requests to user B and send an unlimited number of DMs to B. But the difference is that if the chat request wasn't approved, the B client will never download any of A's DMs.

[arthurfranca]

The client still needs to download all requests (which can be made huge -- 100KB each)

@vitorpamplona I've added suggestion for relays to block chat request events higher than 3KB.

Added PoW requirement to Chat Requests (enforced by the receiving clients). From the user point of view, sender client can send DMs freely (while it is mining the Chat Request PoW in the background).

Receiving client won't ever download any of someone's DMs if the receiving user hasn't approved the specific chat request.

[staab]

Yeah, I don't see a problem worth solving here. Spam will happen anyway, and there are other ways to deal with it than creating something that nominally helps, but really just moves the problem around. The created_at thing is annoying, but Coracle already uses a 1 day overlap when looking for messages and stuff in case there was a delay in them being broadcast. I would be fine with reducing the timestamp randomization, 1 week was "this seems like enough to be pretty safe", but isn't based in science. 1 or 2 days would be an acceptable recommendation for me.

[vitorpamplona]

I don't want to discourage you from trying this new idea, but I agree with @staab that it seems that it doesn't add much.

But maybe we are all wrong and this becomes a huge issue for somebody or in the future (with the spam and stuff). Who knows...

[jb55]

• The receiver has no way to fetch just DMs (Gift-wrap is a generic kind:1059 event)
• The receiver has no way to fetch just Gift-wraps/DMs from a specific person
• The receiver needs to fetch messages from the "last read at" moment minus one week onwards

These have always been dealbreakers for me, the UX is pretty bad without these features. So thanks for looking at alternatives. I will check out this spec when I get the chance.

[arthurfranca]

@vitorpamplona what do you think of #945 (comment) to auto-rotate keys because of the social attack you mentioned?

[vitorpamplona]

It seems better than the current version, but the flexibility of salt is not part of nip44 yet.

[staab]

@arthurfranca why not use 0xchat's NIP 101 key exchange? It's been around for a long time, works with gift wraps, and solves problems 1 and 2.

[arthurfranca]

@staab a big difference from the current state of my PR is that by using the conversation key (CK) as alias for both participants, we don't need to exchange keys at all (just poke the other party with a chat request) cause both users can generate the CK from their own privkey and the other party's pubkey.

If we add vitor's idea of some digits (the number of digits is to be defined on the NIP) of the date as salt when generating the CK we also cover NIP-101's kind:10104 event (Update) but better because it would also be automatic, without an exchange step (just consider the date range the client wants to use to fetch DMs). Funny that while @vitorpamplona don't like this PR, I'm stealing his ideas to enhance it xD

Also with NIP-101 it isn't clear if the alias pubkey will be used inside a p tag (or like here as author filter). Assuming it is to be used as p tag (like regular gift wraps), there is the problem of requiring the other party (User B) to get online to accept the request by sending its own pubkey alias before the user A can start sending DMs. So you gotta buffer DMs somewhere.

NIP-101 forgot to define an event to store the active chat sessions while we picked one (Approved Chats).

There is a chance that NIP-101 kind:10102 (Reject) may be abused to check if someone is online.

[arthurfranca]

@vitorpamplona the added complexity of rotating keys with salt seems small enough to be a good addition.

If I'm listening to live DM events starting at 1704456062178 timestamp, with the below function I know I have to restart it at 1705000000000 with a new set of pubkeys (pubkeys derived from conversation keys corresponding to the 1-1 chat channels I'm tracking).

function getNextWindow (ts = Date.now(), saltDigits = 4) {
  console.log(ts)
  let tsStr = ts.toString()
  const digitsLeft = tsStr.length - saltDigits
  const salt = tsStr.slice(0, saltDigits)
  const rest = tsStr.slice(saltDigits)
  const window = 10 ** digitsLeft
  return ts - rest + window
}

[mikedilger]

I think we do indeed need to mitigate the spam problem of the NIP-17 PR.

This PR looks too long and I can't focus long enough to review the whole thing.

The first major problem here is that conversation keys are not private keys. You cannot just transform any random set of 256 bits into a private key. Some such bit sequences are invalid on the secp256k1 curve. It is not a "safe" curve wherein such operations are safe to do. For example, anything greater than fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141, the number zero, and possibly others (points of order on twists). Although you can almost do this and there might be a way of getting around the edge cases. But given people will have chosen their nostr keys already, we cannot choose a different conversation key to get around it can we?

Furthermore, you could get forward secrecy if you did in fact generate keys and exchange them. Why not strive for forward secrecy?

[staab]

@mikedilger helpful insight, I was suspicious the "conversation key as private key" idea was flawed.

Furthermore, you could get forward secrecy if you did in fact generate keys and exchange them. Why not strive for forward secrecy?

Yes, this is basically what everyone else who does secure messaging does. If people want to make more secure versions of direct messaging, key exchange seems like the way to go, especially since it doesn't require a whole separate standard, just an extension that clients can add on top of an existing messaging standard.

[mikedilger]

I don't think you need a kind 14 for the rumor. I think the rumor can be any kind. But I don't have a strong opinion here.

I don't like sending the private key for deleting. I'd rather if giftwraps were deletable by the p-tagged person. I thought that was part of #716 but I don't see it and I'm asking over there too.

[arthurfranca]

I'd rather if giftwraps were deletable by the p-tagged person

Here we use the same conversation key to sign (gift-wrapped) DMs both when user A is the sender and when user B is the sender; there is no p tag (there's a fake one that I think I should remove). The same DM note can be downloaded by both (using filter { authors: [pubkey-from-conversation-key] }) and the inner event reveals who is the sender.

The first major problem here is that conversation keys are not private keys. You cannot just transform any random set of 256 bits into a private key.

If there is no way around it it's a shame cause the { authors: [pubkey-from-conversation-key] } filter is effectively a private channel between two users, a privkey that just the two users can use to sign things. And the conversation key doesn't need to be exchanged (both users already know how to generate it).

If conversation key can't be used as privkey I need to rewrite this PR to use an alias exchange flow.

[staab]

If conversation key can't be used as privkey I need to rewrite this PR to use an alias exchange flow.

If this is the direction you're going, we should use 0xchat's NIP 101 as mentioned above.

[mikedilger]

If conversation key can't be used as privkey I need to rewrite this PR to use an alias exchange flow.

Before changing the PR, I would rope in @paulmillr to see if there is a way to solve it.

But before doing that, I think we should all try to get on the same page. @staab and @vitorpamplona seem quite happy with #686 and not too persuaded by the spam issue. But I can imagine millions of giftwrap events, all from different public keys so not spam-filterable by the relays, becoming a denial-of-service attack with no real solution other than giving up on giftwraps entirely (e.g. there is no way to distinguish the spam ones from legit giftwraps).

Then again, their PR is elegant and this one is long and klunky. And I'm entirely unconvinced that these two PR choices are the only 2 choices we have. I may have to think up a 3rd way.

[mikedilger]

Why not just put a PoW on #716 gift wraps? Then the sender has to do more work than the recipient who just downloads it and deletes it.

[staab]

PoW is an easy addition if it becomes a problem, although I think most people are skeptical that it would actually work. I'm not convinced by the spam attack problem because it's basically no different from anything else. You could spam regular kind 4's, and they would either have to be filtered out client side or relay side, but in either case someone has to do the analysis to figure out that pubkey a is not a valid contact of pubkey b. That's arguably cheaper than decrypting first then doing it client side, but they're really two versions of the same problem in my book. Anyway, I think inbox write-auth might be a better solution to the spam problem, since even if messages aren't marked as from a particular person, they still would have to prove their legitimacy to the relay before even publishing the spam (auth challenge might be resolved by checking the sender's key, or it could be resolved using some other key that represents a claim, maybe one that the recipient issued).

To elaborate on that, let's say Alice wants to say "I only want messages from Bob". She sends a claim to Bob using NIP 86 with a private key he should use to AUTH with her inbox. When someone wants to send a message to Alice, they have to send it to inbox.alice.com, which says AUTH for me, with that key Alice gave you. Only Bob can do that, so only his messages get delivered.

[mikedilger]

Spam can be filtered by content... except when content is encrypted. Spam can be filtered by how well a pubkey is known and trusted after AUTHing.... except when giftwraps use random pubkeys by design. Giftwraps are the worst case since both the content is encrypted and the pubkey is random, and yet we need to let them all through or they cannot work. And so this is not just another version of the same problem in my book, it is the most extreme case that cannot use any of the spam filtering mechanisms (relay side). No other event kind suffers this fate.

Vitor downloads them all, presumes that if he gets a bunch at once he can just delete them (one of them might have been me), and filters the content on the client. That works for 100. But will it work for 1,000,000? This is a special case that relays are helpless to assist with.

The only thing I can think to do is PoW. Anything else is either too complex, a chicken and egg problem, breaks the privacy model, etc. PoW should work for this particular problem because clients can handle hundreds of these... but they can't handle millions of these. PoW isn't fair with ASICs vs phones, but if the cost is high enough the problem won't eventuate. if the cost is nearly free (as it is now) some asshole is gonna flood us.

[mikedilger]

I've made my points.

nack on this PR.

[arthurfranca]

@mikedilger It's worse to require PoW from DM events cause people may send messages at a high frequency, so it can get slow to send them.

On this PR we move the PoW to the "chat request" step, that is supposed to happen at much lower frequency so the PoW difficulty can start bigger. The PoW difficulty requirement can grow the more a pubkey receives chat requests on a small time frame. It is also an smaller event (no content nor tags) that relays can enforce a max byte size. Another good thing of having a "chat request" (or key/alias exchange) step is that the DM events are fetched just from approved authors (and there is no p tag to spam on DMs).

So the spam can happen only on chat requests, but it's much easier to fight it.

[paulmillr]

Conversation key can’t be used as private key. It needs to be at least 48 bytes to be safely reduceable to secp256k1.

[arthurfranca]

Thx @paulmillr and @mikedilger
I'm going to close this and probably write a new draft PR using random privkeys as aliases.

Imo we need to assess new options. Clearly #686 has spam problems at the current version.

[arthurfranca]

New PR without using conversation key as privkey: #978