store: Preallocate output buffer when encoding postings.#2812
Merged
bwplotka merged 3 commits intothanos-io:masterfrom Jun 26, 2020
pstibrany:preallocate-encoding-buffer
Merged
store: Preallocate output buffer when encoding postings.#2812bwplotka merged 3 commits intothanos-io:masterfrom pstibrany:preallocate-encoding-buffer
bwplotka merged 3 commits intothanos-io:masterfrom
pstibrany:preallocate-encoding-buffer
Conversation
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
pracucci
approved these changes
Jun 26, 2020
Contributor
pracucci
left a comment
There was a problem hiding this comment.
Results are amazing! Good job 👏 I left a minor nit, but no blocker to me.
bwplotka
approved these changes
Jun 26, 2020
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Contributor
Author
Indeed :) |
Contributor
Author
|
Test and doc failures seem to be unrelated to my changes :-( |
Member
|
Thanks! |
paulfantom
added a commit
to paulfantom/thanos
that referenced
this pull request
Jul 9, 2020
* upstream/release-0.14: (46 commits) Cut release v0.14.0-rc.1 (thanos-io#2853) Query: correctly marshal errors to JSON and ignore if nil (thanos-io#2848) ci: Manually download promu in crossbuild stage (thanos-io#2828) Cut release v0.14.0-rc.0 (thanos-io#2826) Soft cut changelog on master to indicate v0.14.0 being in progress (thanos-io#2824) Update ThanosReceiveNoUpload to select sum == 0 (thanos-io#2819) receive: Added more observability, fixed leaktest, to actually check leaks ): (thanos-io#2817) Query: always return a string in the `lastError` field (thanos-io#2809) Added missing CHANGELOG entry for PR 2613 (thanos-io#2820) receive: Fixed small options race; Removed unused StartTime feature. (thanos-io#2816) go.mod: Bump Prometheus to current latest (thanos-io#2814) Implement CLI Flags page in React UI (thanos-io#2796) Improve ThanosReceiveNoUpload to only alert on current instances store: Preallocate output buffer when encoding postings. (thanos-io#2812) compact: introduce flag --block-viewer.global.sync-block-interval (thanos-io#2752) docs: compact: add blurb about how retention policy works (thanos-io#2808) Reduced memory allocations in readIndexRange() (thanos-io#2807) ui: Add Stores page to React UI (thanos-io#2754) Added Kemal to Maintainer Role; Kemal is volounteering to be next release shephard (thanos-io#2804) proposal: Add scalable rule storage proposal (thanos-io#2661) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR modifies
diffVarintSnappyEncodemethod to preallocate output buffer based on number of passed postings. This helps to avoid reallocations.Notes for reviewers: I chose to pass expected number of postings as another parameter. Alternative I considered was introducing an interface with
PostingsCountmethod, which may look nicer, but would add hidden dependency intodiffVarintSnappyEncode-- for postings implementing the interface, it would be faster (use less allocs) than for postings not implementing it. Passing extra parameter is more explicit.Verification
Added benchmark.