Skip to content

store: Postings fetching optimizations #2294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bwplotka merged 19 commits into from
Mar 23, 2020
Merged

store: Postings fetching optimizations #2294

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
62ef666
Avoid fetching duplicate keys.
pstibrany Mar 19, 2020
b80fcc4
Added shortcuts
pstibrany Mar 19, 2020
280db8c
Optimize away fetching of ALL postings, if possible.
pstibrany Mar 19, 2020
701d88e
Don't do individual index.Without, but merge them first.
pstibrany Mar 19, 2020
02d87c9
Don't use map for fetching postings, but return slice instead.
pstibrany Mar 19, 2020
f3e1656
Renamed 'all' to 'allRequested'.
pstibrany Mar 19, 2020
ae3f5af
Typo
pstibrany Mar 19, 2020
00b84f6
Make linter happy.
pstibrany Mar 19, 2020
b6f44ee
Added comment to fetchPostings.
pstibrany Mar 20, 2020
68ba009
Group vars
pstibrany Mar 21, 2020
0248583
Comments
pstibrany Mar 21, 2020
6d8c48c
Use allPostings and emptyPostings variables for special cases.
pstibrany Mar 21, 2020
f23eb71
Unify terminology to "special All postings"
pstibrany Mar 21, 2020
c784dda
Address feedback.
pstibrany Mar 21, 2020
5a13b8b
Added CHANGELOG.md entry.
pstibrany Mar 21, 2020
7122161
Fix check for empty group.
pstibrany Mar 21, 2020
658091d
Comment
pstibrany Mar 21, 2020
029d301
Special All postings is now added as a new group
pstibrany Mar 23, 2020
d9396a9
Updated comment
pstibrany Mar 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Renamed 'all' to 'allRequested'. 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
  • Loading branch information
@pstibrany
pstibrany committed Mar 19, 2020
commit f3e16562d6a3e484a70af3166f944dabc0919ad6
6 changes: 3 additions & 3 deletions pkg/store/bucket.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1312,7 +1312,7 @@ func newBucketIndexReader(ctx context.Context, block *bucketBlock) *bucketIndexR
func (r *bucketIndexReader) ExpandedPostings(ms []*labels.Matcher) ([]uint64, error) {
var postingGroups []*postingGroup

all := false
allRequested := false
hasAdds := false
keys := []labels.Label(nil)

Expand All @@ -1330,7 +1330,7 @@ func (r *bucketIndexReader) ExpandedPostings(ms []*labels.Matcher) ([]uint64, er
}

postingGroups = append(postingGroups, pg)
all = all || pg.addAll
allRequested = allRequested || pg.addAll
hasAdds = hasAdds || len(pg.addKeys) > 0

// postings returned by fetchPostings will be in the same order as keys
Expand All @@ -1346,7 +1346,7 @@ func (r *bucketIndexReader) ExpandedPostings(ms []*labels.Matcher) ([]uint64, er

allKeyIndex := -1
// we only need All postings if there are no other adds. If there are, we can skip fetching ALL postings completely.
if all && !hasAdds {
if allRequested && !hasAdds {
// remember the index (will be used later as a flag, and also to access postings),
// and ask fetchPostings to fetch ALL postings too
allKeyIndex = len(keys)
Expand Down