Add skipChunk option in series API by yeya24 · Pull Request #1904 · thanos-io/thanos

yeya24 · 2019-12-18T03:09:38Z

Signed-off-by: yeya24 yb532204897@gmail.com

I added CHANGELOG entry for this change.
Change is not relevant to the end user.

Changes

Fixes #1822 #1774
Add a no-chunk bool in series API to return labels instead of the whole series. Only in querier /api/v1/series API, this option will set to true, otherwise it set to false.

Verification

I have already added test cases for all stores.

For the response time using no-chunk, I tested in my local env.

Using Thanos v0.9.0

curl -sw '%{time_total} %{size_download}\n' -o /dev/null 'http://localhost:10904/api/v1/series?' -d 'match[]={job="prometheus"}'
0.297961 81680

Using this PR

curl -sw '%{time_total} %{size_download}\n' -o /dev/null 'http://localhost:10908/api/v1/series?' -d 'match[]={job="prometheus"}'
0.024901 81680

GiedriusS · 2019-12-18T17:09:20Z

CHANGELOG.md

 ### Added
 - [#1852](https://github.com/thanos-io/thanos/pull/1852) Add support for `AWS_CONTAINER_CREDENTIALS_FULL_URI` by upgrading to minio-go v6.0.44
 - [#1854](https://github.com/thanos-io/thanos/pull/1854) Update Rule UI to support alerts count displaying and filtering.
+- [#1904](https://github.com/thanos-io/thanos/pull/1904) Add a no-chunk option in Store Series API.


Perhaps lets mention that this greatly improves performance of the /api/v1/series endpoint?

GiedriusS · 2019-12-18T17:12:42Z

pkg/store/proxy_test.go


 			s := newStoreSeriesServer(context.Background())

+			if tc.req.NoChunk == true {


You probably forgot to remove this? :P

GiedriusS · 2019-12-18T17:15:54Z

pkg/store/prometheus.go

+	q.Add("start", string(startTime))
+	q.Add("end", string(endTime))
+
+	req, err := http.NewRequest("GET", u.String(), nil)


You can use http.NewRequestWithContext here to shorten things :P

Also... can we use POST here to avoid potential character limits with big queries?

Oh, make sense! Thanks 😄

I recheck the POST thing. Looks like https://github.com/prometheus/client_golang/blob/master/api/prometheus/v1/api.go also uses GET. So is it not necessary to use POST?

I would say we need to do GET to be fully compatible with old pre 2.8.x Prometheus versions.

GiedriusS · 2019-12-18T17:22:34Z

pkg/store/matchers.go

+		return m.Name + `!~"` + m.Value + `"`
+	}
+
+	return ""


Maybe lets panic here so that it would act as an assertion that this for sure cannot be reached?

TBH, I am not sure whether there is a case to not match anything? Since the matchers parameters are passed from HTTP request, so it will validate the user input to ensure its validity

GiedriusS · 2019-12-18T17:31:05Z

pkg/store/matchers.go

+	return "{" + res + "}"
+}
+
+func labelMatcherToString(m storepb.LabelMatcher) string {


Maybe lets reuse https://github.com/thanos-io/thanos/blob/master/pkg/store/matchers.go#L9 here + https://godoc.org/github.com/prometheus/tsdb/labels#Matcher ?

wow... I didn't even find this is already implemented 😢 Thanks, I will switch to it.

GiedriusS

Besides a few things pretty good work! Thank you a lot!

bwplotka

Nice, I think I am generally ok with this 👍

bwplotka · 2019-12-20T18:32:07Z

pkg/store/prometheus.go

+	u.Path = path.Join(u.Path, "/api/v1/series")
+	q := u.Query()
+
+	metric, err := matchersToMetric(matchers)


both are matchers, string vs protobuf, right? Can we name accordingly?

bwplotka · 2019-12-20T18:36:17Z

pkg/store/prometheus.go

+	q.Add("start", string(startTime))
+	q.Add("end", string(endTime))
+
+	req, err := http.NewRequest("GET", u.String(), nil)


I would say we need to do GET to be fully compatible with old pre 2.8.x Prometheus versions.

bwplotka · 2019-12-20T18:37:20Z

pkg/store/prometheus.go

+	defer runutil.ExhaustCloseWithLogOnErr(p.logger, resp.Body, "series request body")
+
+	if resp.StatusCode/100 != 2 {
+		return nil, status.Error(codes.Internal, fmt.Sprintf("request Prometheus server failed, code %s", resp.Status))


Suggested change

return nil, status.Error(codes.Internal, fmt.Sprintf("request Prometheus server failed, code %s", resp.Status))

return nil, status.Errorf(codes.Internal, "request Prometheus server failed, code %s", resp.Status)

Still to do (:

bwplotka · 2019-12-20T19:03:57Z

pkg/store/storepb/rpc.proto

  // TODO(bwplotka): Move Thanos components to use strategy instead. Including QueryAPI.
  PartialResponseStrategy partial_response_strategy = 7;
+
+  bool no_chunk = 8;


This requires a comment.

Overall are we happy with naming and this API field? Wonder how extendible this is. Overall having bool field to RPC is sometimes a code/design smell.

I think I am ok with this here as it's a caller decision to no skip chunks so maybe field should be named:

bool skip_chunks or bool discard_chunks? (:

I prefer skip_chunks. Do we need to make this as an enum? Currently I don't see some possible extensions of this field.

bwplotka

Thanks! Some comments, small nits, otherwise LGTM!

bwplotka · 2020-01-03T10:45:57Z

pkg/store/bucket_e2e_test.go

-				MaxTime: maxt,
+				MinTime:    mint,
+				MaxTime:    maxt,
+				SkipChunks: false,


false statements we can skip as it's a default (:

bwplotka · 2020-01-03T10:48:39Z

pkg/store/matchers.go

+
+	for _, m := range matchers {
+		if m.Name == "__name__" {
+			nameMatchers = append(nameMatchers, m)


Do we need to separate those? I believe {__name__=..., ...} form will work just fine.

See http://demo.robustperception.io:9090/api/v1/series?match[]={__name__=%22up%22}

bwplotka · 2020-01-03T10:58:48Z

pkg/store/prometheus.go

+	defer runutil.ExhaustCloseWithLogOnErr(p.logger, resp.Body, "series request body")
+
+	if resp.StatusCode/100 != 2 {
+		return nil, status.Error(codes.Internal, fmt.Sprintf("request Prometheus server failed, code %s", resp.Status))


Still to do (:

bwplotka · 2020-01-03T10:59:06Z

pkg/store/prometheus_test.go

 			Matchers: []storepb.LabelMatcher{
 				{Type: storepb.LabelMatcher_EQ, Name: "a", Value: "b"},
 			},
+			SkipChunks: false,


ditto, false is a default

Signed-off-by: yeya24 <yb532204897@gmail.com> add e2e tests Signed-off-by: yeya24 <yb532204897@gmail.com> add tests for all stores Signed-off-by: yeya24 <yb532204897@gmail.com> add changelog Signed-off-by: yeya24 <yb532204897@gmail.com>

yeya24 · 2020-01-03T14:28:41Z

@bwplotka Thanks for your detailed review. I have addressed all of your comments.

bwplotka

LGTM 👍

Thanks!

bwplotka · 2020-01-03T20:03:04Z

pkg/store/matchers.go

+	}
+
+	for i, m := range matchers {
+		res += m.String()


Generally, we do strings.Join but this is fine for now (:

yeya24 force-pushed the no-chunk branch 5 times, most recently from e5cca30 to f9d59aa Compare December 18, 2019 15:29

GiedriusS reviewed Dec 18, 2019

View reviewed changes

yeya24 force-pushed the no-chunk branch 2 times, most recently from 547467b to 80c1972 Compare December 18, 2019 19:05

bwplotka reviewed Dec 20, 2019

View reviewed changes

yeya24 force-pushed the no-chunk branch 3 times, most recently from 3b3c620 to 62882bc Compare December 23, 2019 14:39

bwplotka approved these changes Jan 3, 2020

View reviewed changes

yeya24 force-pushed the no-chunk branch from 62882bc to a68ee40 Compare January 3, 2020 14:26

add no-chunk option in series API

9c8f29e

Signed-off-by: yeya24 <yb532204897@gmail.com> add e2e tests Signed-off-by: yeya24 <yb532204897@gmail.com> add tests for all stores Signed-off-by: yeya24 <yb532204897@gmail.com> add changelog Signed-off-by: yeya24 <yb532204897@gmail.com>

yeya24 force-pushed the no-chunk branch from a68ee40 to 9c8f29e Compare January 3, 2020 14:27

yeya24 changed the title ~~Add no-chunk option in series API~~ Add skipChunk option in series API Jan 3, 2020

bwplotka approved these changes Jan 3, 2020

View reviewed changes

pkg/store/matchers.go

}

for i, m := range matchers {

res += m.String()

Copy link

Member

bwplotka Jan 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, we do strings.Join but this is fine for now (:

bwplotka merged commit d9764fb into thanos-io:master Jan 3, 2020

yeya24 deleted the no-chunk branch January 3, 2020 20:28

IKSIN mentioned this pull request Jan 20, 2020

query: error in #1904 - /series with SkipChunks not use start/end args. #2011

Closed


		s := newStoreSeriesServer(context.Background())

		if tc.req.NoChunk == true {

	return nil, status.Error(codes.Internal, fmt.Sprintf("request Prometheus server failed, code %s", resp.Status))
	return nil, status.Errorf(codes.Internal, "request Prometheus server failed, code %s", resp.Status)

Conversation

yeya24 commented Dec 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Verification

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yeya24 Dec 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GiedriusS Dec 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GiedriusS left a comment

Choose a reason for hiding this comment

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yeya24 commented Jan 3, 2020

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

yeya24 commented Dec 18, 2019 •

edited

Loading

yeya24 Dec 18, 2019 •

edited

Loading

GiedriusS Dec 18, 2019 •

edited

Loading