Skip to content

Vertical Block Sharding Proposal #4191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Biswajitghosh98 wants to merge 8 commits into
base: cap-size-com
Choose a base branch
from
Open

Vertical Block Sharding Proposal #4191

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
11ddafa
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 6, 2021
5de0618
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 6, 2021
c41d3ff
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 6, 2021
99429e9
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 10, 2021
d4d511d
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 11, 2021
a18ad22
Add files via upload
Biswajitghosh98 May 11, 2021
c88e9a4
Update 202011_vertical_block_sharding.md
Biswajitghosh98 May 11, 2021
68e3413
Update docs/proposals/202011_vertical_block_sharding.md
Biswajitghosh98 May 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update 202011_vertical_block_sharding.md
  • Loading branch information
@Biswajitghosh98
Biswajitghosh98 authored May 6, 2021
commit c41d3ffe2eb0791d0eb30b36dec50803ab0b2daf
2 changes: 1 addition & 1 deletion docs/proposals/202011_vertical_block_sharding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ index size).

**We propose to vertically shard all blocks which goes beyond the specified index size limit during the compaction process, else efficiently shard the blocks at compaction level 0.**

* Special metadata in meta.json indicating a shard.
* We'll be using a special metadata in meta.json indicating a shard.
Copy link
Contributor

@fpetkovski fpetkovski Jun 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be another external label?

Copy link
Contributor

@yeya24 yeya24 Jun 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think hash_number and shard_level are external labels as mentioned below.


#### Approach 1: Sharding at compaction level 0

Expand Down