Skip to content

[ADR] JMAP: Avoid ElasticSearch on critical reads #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
chibenwa wants to merge 13 commits into from
Closed

[ADR] JMAP: Avoid ElasticSearch on critical reads #259

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8deb190
[ADR] JMAP: Avoid ElasticSearch on critical reads
chibenwa Nov 11, 2020
a4ae094
fixup! [ADR] JMAP: Avoid ElasticSearch on critical reads
chibenwa Nov 12, 2020
532cccf
fixup! fixup! [ADR] JMAP: Avoid ElasticSearch on critical reads
chibenwa Nov 12, 2020
bc12fbc
fixup! fixup! fixup! [ADR] JMAP: Avoid ElasticSearch on critical reads
chibenwa Nov 13, 2020
5a3640f
fixup! fixup! fixup! fixup! [ADR] JMAP: Avoid ElasticSearch on critic…
chibenwa Nov 13, 2020
2fe64cb
Update src/adr/0043-avoid-elasticsearch-on-critical-reads.md
chibenwa Nov 13, 2020
25c7d92
Update src/adr/0043-avoid-elasticsearch-on-critical-reads.md
chibenwa Nov 13, 2020
70f7898
Update src/adr/0043-avoid-elasticsearch-on-critical-reads.md
chibenwa Nov 13, 2020
5a03c80
Update src/adr/0043-avoid-elasticsearch-on-critical-reads.md
chibenwa Nov 13, 2020
892da60
fixup! Update src/adr/0043-avoid-elasticsearch-on-critical-reads.md
chibenwa Nov 13, 2020
3907a83
[ADR] JMAP: Mention InfiniSpan as a possible alternative
chibenwa Nov 16, 2020
a95aa3d
fixup! [ADR] JMAP: Avoid ElasticSearch on critical reads
chibenwa Nov 16, 2020
f8d1b5d
fixup! [ADR] JMAP: Mention InfiniSpan as a possible alternative
chibenwa Nov 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/adr/0043-avoid-elasticsearch-on-critical-reads.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,4 +156,6 @@ PRIMARY KEY mailboxId
CLUSTERING COLUMN messageId
COLUMN sentAt
COLUMN receivedAt
```
```

Note that to handle position & limit, we need to fetch `position + limit` ordered items then removing `position` firsts items.
Copy link
Member

@rouazana rouazana Nov 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so if I scroll quickly n times, I will generate 1+2+...+n = n*(n+1)/2 cassandra requests ~= O(n²)

that's pretty bad, no? Couldn't it be a cause of ElasticSearch slowness? Could it slow down Cassandra?

Copy link
Contributor Author

@chibenwa chibenwa Nov 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True for Cassandra.

True for ElasticSearch.

JMAP includes some limits concurent call, rate limiting - that can help mitigating these concenrs in the future.

Couldn't it be a cause of ElasticSearch slowness?

Maybe for some.

I succeeded to clearly link some to reindexing as well thanks to @tuanlc .