Skip to content

Proposal for Thanos Sharding.#1501

Merged
bwplotka merged 1 commit intomasterfrom
func-sharding
Oct 8, 2019
Merged

Proposal for Thanos Sharding.#1501
bwplotka merged 1 commit intomasterfrom
func-sharding

Conversation

@bwplotka
Copy link
Member

@bwplotka bwplotka commented Sep 7, 2019

Signed-off-by: Bartek Plotka bwplotka@gmail.com

cc @jojohappy @mattrco @metalmatze

@brancz
Copy link
Member

brancz commented Sep 9, 2019

Generally looks good, just some grammar and one consistency/cleanup type question.

Copy link
Member

@GiedriusS GiedriusS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments by @brancz look on point and the proposal LGTM besides one question. Thanks! This is definitely needed and will be very nice.

Copy link
Member

@jojohappy jojohappy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! LGTM! 👍

@bwplotka
Copy link
Member Author

This is not ready - I need to rework this to simplify and focus on sharding only but explain how it can be extended for more advanced use cases if needed.

* `<name>`
* `__block_objstore_bucket_endpoint`
* `__block_objstore_bucket_name`
* `__block_objstore_bucket_path`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit confusing to me. So let's say there will be a special __block_objstore_bucket_path label, how can we manipulate it?

As I understand it we will be able to configure Thanos Store gateway to filter out folders from s3?

  action: drop
  regex: "folder_name"
  source_labels:
  - __block_objstore_bucket_path

Not sure how we can we do with __block_objstore_bucket_endpoint, __block_objstore_bucket_name as they will be constant on the time?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for now the options are limited - maybe indeed we should limit this (:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can talk about those details on the PR I guess.

@jojohappy jojohappy mentioned this pull request Sep 30, 2019
3 tasks
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
@bwplotka
Copy link
Member Author

bwplotka commented Oct 8, 2019

This should be good to go!

@bwplotka bwplotka merged commit 4c1761a into master Oct 8, 2019
@bwplotka bwplotka deleted the func-sharding branch October 8, 2019 21:55
GiedriusS pushed a commit that referenced this pull request Oct 28, 2019
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants