-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Open
13 / 1913 of 19 issues completedLabels
A-dbRelated to the databaseRelated to the databaseA-static-filesRelated to static filesRelated to static filesC-debtA clean up/refactor of existing codeA clean up/refactor of existing codeC-enhancementNew feature or requestNew feature or requestC-tracking-issueAn issue that collects information about a broad development initiativeAn issue that collects information about a broad development initiative
Milestone
Description
Describe the feature
Currently, Reth uses two storage systems for all data: MDBX database for hot data, and Nippy Jar static files for cold data. It works the following way:
- When new block is received, executed, and validated, we insert headers, transactions, and receipts into both database and static files.
- When reading headers/transactions/receipts, we look in both database and static files, in case if the data is present only in database.
- Pruner deletes the data from database, leaving it only in static files.
This doesn't make much sense now, but before we didn't write to static files directly, and instead were writing only to database, then StaticFileProducer was copying the data to static files, and Pruner removing it from database. Since the time we started writing to static files immediately on committing new block, we can simplify the whole storage system a lot.
The main goals of this tracking issue are:
- Do not ever have duplicate data between database and static files. This simplifies the mental model of storage, and allows to look into only one storage in providers.
- Move all append-only data to static files. This leaves only hot ever-changing data and indices in the database, making the access to historical data faster and the size smaller.
Additional context
No response
emhane
Sub-issues
Metadata
Metadata
Assignees
Labels
A-dbRelated to the databaseRelated to the databaseA-static-filesRelated to static filesRelated to static filesC-debtA clean up/refactor of existing codeA clean up/refactor of existing codeC-enhancementNew feature or requestNew feature or requestC-tracking-issueAn issue that collects information about a broad development initiativeAn issue that collects information about a broad development initiative
Type
Projects
Status
In Progress