Skip to content

store: Postings fetching optimizations #2294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bwplotka merged 19 commits into from
Mar 23, 2020
Merged

store: Postings fetching optimizations #2294

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
62ef666
Avoid fetching duplicate keys.
pstibrany Mar 19, 2020
b80fcc4
Added shortcuts
pstibrany Mar 19, 2020
280db8c
Optimize away fetching of ALL postings, if possible.
pstibrany Mar 19, 2020
701d88e
Don't do individual index.Without, but merge them first.
pstibrany Mar 19, 2020
02d87c9
Don't use map for fetching postings, but return slice instead.
pstibrany Mar 19, 2020
f3e1656
Renamed 'all' to 'allRequested'.
pstibrany Mar 19, 2020
ae3f5af
Typo
pstibrany Mar 19, 2020
00b84f6
Make linter happy.
pstibrany Mar 19, 2020
b6f44ee
Added comment to fetchPostings.
pstibrany Mar 20, 2020
68ba009
Group vars
pstibrany Mar 21, 2020
0248583
Comments
pstibrany Mar 21, 2020
6d8c48c
Use allPostings and emptyPostings variables for special cases.
pstibrany Mar 21, 2020
f23eb71
Unify terminology to "special All postings"
pstibrany Mar 21, 2020
c784dda
Address feedback.
pstibrany Mar 21, 2020
5a13b8b
Added CHANGELOG.md entry.
pstibrany Mar 21, 2020
7122161
Fix check for empty group.
pstibrany Mar 21, 2020
658091d
Comment
pstibrany Mar 21, 2020
029d301
Special All postings is now added as a new group
pstibrany Mar 23, 2020
d9396a9
Updated comment
pstibrany Mar 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Typo 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
  • Loading branch information
@pstibrany
pstibrany committed Mar 19, 2020
commit ae3f5af32d5aa99c8309187f7c5ae1f233925c7f
2 changes: 1 addition & 1 deletion pkg/store/bucket.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1324,7 +1324,7 @@ func (r *bucketIndexReader) ExpandedPostings(ms []*labels.Matcher) ([]uint64, er
return nil, errors.Wrap(err, "toPostingGroup")
}

// interesction would return no postings anyway
// intersection would return no postings anyway
if pg.alwaysEmptyPostings() {
Copy link
Member

@bwplotka bwplotka Mar 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if pg.alwaysEmptyPostings() {
if pg.IsEmptyPostings() {

Copy link
Contributor Author

@pstibrany pstibrany Mar 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IsEmptyPostings is a different guarantee... there can be labels that are not present in the block, and will also return empty postings. IsEmptyPostings name would suggest that it returns true in such case, but it doesn't. On the other hand, IsAlwaysEmptyPostings (new suggested name) says that this will be empty no matter the block.

Copy link
Contributor Author

@pstibrany pstibrany Mar 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this method in favor of checking for specific emptyPostingsGroup sentinel value.

Copy link
Contributor Author

@pstibrany pstibrany Mar 21, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And now removed check for sentinel value again, since it's incorrect. We really need to check if group is empty. There are other ways how group can be empty, e.g. by using label="non-existant-value" matcher.

return nil, nil
}
Expand Down