-
Notifications
You must be signed in to change notification settings - Fork 304
Enable the CacheLibWrapper class as a RocksDB Plugin #184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mrambacher
wants to merge
12
commits into
facebook:main
Choose a base branch
from
mrambacher:CachelibRocksDB
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
00c775a
First pass at adding Cachelib as Pluggable Secondary Cache
mrambacher fea3f81
Next pass at Cachelib/Rocks integration
mrambacher 6130377
Add namespace
mrambacher 8392fb2
Attempt to fix test to build
mrambacher 7f3b1a1
Another build change
mrambacher ec1186f
Next pass at CacheLib as a SecondaryCache
mrambacher 143ac0d
Fix bad var names
mrambacher d825c15
Fix test
mrambacher 75a540c
Cleanup tests
mrambacher 28a8ae3
Add README, fix test issue
mrambacher 52e2d8e
Fix typos
mrambacher 7e62abf
Merge remote-tracking branch 'upstream/main' into CachelibRocksDB
mrambacher File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add README, fix test issue
- Loading branch information
commit 28a8ae3acbf024edfe2ebb4977077142edcc876e
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # RocksDB CacheLib Secondary Cache | ||
|
|
||
| This directory contains files and tests for building Cachelib as a SecondaryCache for RocksDB. | ||
|
|
||
| # Build | ||
|
|
||
| Currently the build is only supported via cmake. | ||
|
|
||
| First, build cachelib through the normal build procedure. | ||
|
|
||
| To build under RocksDB, link this directory adaptor/rocksdb_secondary_cache) to the plugins/cachelib directory under RocksDB: | ||
| ``` | ||
| $ln -s .../secondary_cache .../plugins/cachelib). | ||
| ``` | ||
| This will allow RocksDB to find and build the CacheLib plugin code. | ||
|
|
||
| Next, under the RocksDB build directory, instruct RocksDB to build and include the cachelib plugin: | ||
| ``` | ||
| $cmake -DROCKSDB_PLUGINS=cachelib -DCMAKE_PREFIX_PATH=<Path to CacheLib> -DCMAKE_MODULE_PATH="<Path to CacheLib>" -DWITH_GFLAGS=OFF .. | ||
| ``` | ||
| where the prefix path points to the opt/cachelib directory and module path points to the cachelib/cmake directory. | ||
|
|
||
| Finally, build RocksDB using "make". | ||
|
|
||
| # Using the RocksDB Cachelib Secondary Cache | ||
|
|
||
| The Secondary Cache can be created via either the SecondaryCache::CreateFromString or NewRocksCachelibWrapper APIs. | ||
| ``` | ||
| SecondaryCache::CreateFromString(..., "id=RocksCachelibWrapper; ..."); | ||
|
|
||
| RoksCachelibOptions options; | ||
| NewRocksCachelibWrapper(options); | ||
| ``` | ||
|
|
||
| When using CreateFromString API, the options can be specified as name-value pairs on the command line. The mapping between the field names and options is found in CacheLibWrapper.cpp. | ||
|
|
||
| # Issues and TODO | ||
|
|
||
| * There are incompatibilities between RocksDB and Cachelib on the version of GTEST required and supported. In order to successfully build, you must use the GTEST version in Cachelib and remove the gtest from the RocksDB build. This requirement is true whether or not you are building tests. In the RocksDB build, you must remove GTEST from the "include_directories" and "add_subdirectories" and add " -Wno-error=deprecated-declarations" to the CMAKE_CXX_FLAGS. You may also need to add "-Wno-error=sign-compare" to build the RocksDB tests, depending on your compiler. | ||
|
|
||
| * RocksDB does not currently enable plugins to build their own tests (see PR11052). When the CachelibWrapperTest is added, additional link libraries may be required (See the CMakeLists.txt file). No investigation as to the reason has been undertaken. | ||
|
|
||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm curious, how is the folly dependency handled? Should the user do this first - https://github.com/facebook/rocksdb/wiki/RocksDB-Contribution-Guide#build-rocksdb-with-folly-integration-enabled?