Skip to content

seanmcgary/rustdata

 
 

Repository files navigation

rustdata

Disclaimer

This is an unscientific comparision of Parquet and Postgress for a simple stat calculation.

The dataset is 2500 rows of Ethereum Event data. Eigenlayer IRewardsCoordinator.RewardsClaimed. The data is stored to the database and then serialized back with the purpose of computing some common statistics, mean, min, max, etc.

Postgres

1. Start Postgres database

docker run --name my-postgres -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=password -e POSTGRES_DB=edb -p 5432:5432 -d postgres

2. Upload Postgres data

cargo run --release --bin pg

Parquet

1. run the client

cargo run --release --bin pq

Comparison Chart

Storage Write Time (us) Read Time (us) Storage Size (kB)
Parquet 2687 339 583
Postgres 1037531 2539 1048

About

Example data ingestion and processing in rust

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 98.6%
  • Dockerfile 1.4%