Skip to content

Conversation

@brooksprumo
Copy link

@brooksprumo brooksprumo commented Mar 25, 2024

Problem

There's not a bench in place to compare the perf between append vecs and tiered storage.

Summary of Changes

Add a bench for writing accounts. Subsequent PRs can add benches for reading accounts.

Here's what a run looks like:

write_accounts_file/append_vec/1
                        time:   [966.62 ns 983.87 ns 999.72 ns]
                        thrpt:  [1.0003 Melem/s 1.0164 Melem/s 1.0345 Melem/s]
write_accounts_file/hot_storage/1
                        time:   [277.20 ns 303.86 ns 342.69 ns]
                        thrpt:  [2.9181 Melem/s 3.2910 Melem/s 3.6075 Melem/s]
write_accounts_file/append_vec/100
                        time:   [16.793 µs 16.946 µs 17.084 µs]
                        thrpt:  [5.8533 Melem/s 5.9010 Melem/s 5.9550 Melem/s]
write_accounts_file/hot_storage/100
                        time:   [21.794 µs 21.994 µs 22.174 µs]
                        thrpt:  [4.5098 Melem/s 4.5468 Melem/s 4.5885 Melem/s]
write_accounts_file/append_vec/1000
                        time:   [160.72 µs 161.89 µs 162.82 µs]
                        thrpt:  [6.1419 Melem/s 6.1771 Melem/s 6.2220 Melem/s]
write_accounts_file/hot_storage/1000
                        time:   [222.95 µs 223.76 µs 224.42 µs]
                        thrpt:  [4.4559 Melem/s 4.4691 Melem/s 4.4854 Melem/s]
write_accounts_file/append_vec/10000
                        time:   [1.6278 ms 1.6291 ms 1.6305 ms]
                        thrpt:  [6.1331 Melem/s 6.1384 Melem/s 6.1432 Melem/s]
write_accounts_file/hot_storage/10000
                        time:   [2.2862 ms 2.2894 ms 2.2933 ms]
                        thrpt:  [4.3605 Melem/s 4.3679 Melem/s 4.3741 Melem/s]

@brooksprumo brooksprumo self-assigned this Mar 25, 2024
@brooksprumo brooksprumo force-pushed the bench/accounts-file branch from e514a17 to dade00c Compare March 25, 2024 19:59
Results:

```
write_accounts_file/append_vec/1
                        time:   [966.62 ns 983.87 ns 999.72 ns]
                        thrpt:  [1.0003 Melem/s 1.0164 Melem/s 1.0345 Melem/s]
write_accounts_file/hot_storage/1
                        time:   [277.20 ns 303.86 ns 342.69 ns]
                        thrpt:  [2.9181 Melem/s 3.2910 Melem/s 3.6075 Melem/s]
write_accounts_file/append_vec/100
                        time:   [16.793 µs 16.946 µs 17.084 µs]
                        thrpt:  [5.8533 Melem/s 5.9010 Melem/s 5.9550 Melem/s]
write_accounts_file/hot_storage/100
                        time:   [21.794 µs 21.994 µs 22.174 µs]
                        thrpt:  [4.5098 Melem/s 4.5468 Melem/s 4.5885 Melem/s]
write_accounts_file/append_vec/1000
                        time:   [160.72 µs 161.89 µs 162.82 µs]
                        thrpt:  [6.1419 Melem/s 6.1771 Melem/s 6.2220 Melem/s]
write_accounts_file/hot_storage/1000
                        time:   [222.95 µs 223.76 µs 224.42 µs]
                        thrpt:  [4.4559 Melem/s 4.4691 Melem/s 4.4854 Melem/s]
write_accounts_file/append_vec/10000
                        time:   [1.6278 ms 1.6291 ms 1.6305 ms]
                        thrpt:  [6.1331 Melem/s 6.1384 Melem/s 6.1432 Melem/s]
write_accounts_file/hot_storage/10000
                        time:   [2.2862 ms 2.2894 ms 2.2933 ms]
                        thrpt:  [4.3605 Melem/s 4.3679 Melem/s 4.3741 Melem/s]
```
@brooksprumo brooksprumo force-pushed the bench/accounts-file branch from dade00c to 95fc9d3 Compare March 25, 2024 20:34
@brooksprumo brooksprumo marked this pull request as ready for review March 26, 2024 13:00
Copy link

@jeffwashington jeffwashington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yhchiang-sol
Copy link

Looks good! Thanks for working on this!

@brooksprumo brooksprumo merged commit b2b159a into anza-xyz:master Mar 26, 2024
@brooksprumo brooksprumo deleted the bench/accounts-file branch March 26, 2024 14:44
OliverNChalk pushed a commit to OliverNChalk/agave that referenced this pull request Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants