Removes the storage recycler #118

brooksprumo · 2024-03-06T21:10:00Z

Problem

The storage recycler is no longer necessary. It is the cause of this fastboot issue¹, and prevents this fastboot usage².

Summary of Changes

Remove the storage recycler

More information

In the PR that added the recycler³, it mentions two motivations:

file/mmap creation can be expensive - 100s of ms
mmap deletion causes a flush to disk, which is unnecessary if the data is not needed

Now that we have the accounts write cache, only roots are written to storages. IOW, we don't have storages for slots on a minority fork that got discarded. We also don't "append" to (non-ancient) append vecs anymore. I think this means we won't ever flush a non-root, and we won't have unflushed data in a storage.

I posted graphs of metrics over on Discord⁴ that showed performance metrics w.r.t. dropping storages and creating storages. There are more metrics at the end of this description too. Tl;dr: occasionally this PR takes a while to create a new mmap file. Mmap deletion seems to be faster with this PR.

I think this is an overall win. The code gets simpler. We unlock a desirable use-case. And we can remove 'atomic' from the AccountStorageEntry slot and id fields. Creating an mmap did get slower, once out of an hour window. This is in the background, and the overall flush/clean/shrink metrics did not seem to be negatively impacted.

More Metrics

There are two nodes in these metrics
- brux is this PR, aka recycler OFF
- dev7 is master (the commit prior to this PR), aka recycler ON
This is on mnb, over 1 hour

The number of dropped storages, with the values summed. Both nodes appear the same.

Time to create storages, during shrink. Also the sum. (I have 'min' and 'max' graphs too, but they look similar.) Presumably this is when recycler ON should be a big win. We do see that this does spike when the recycler is OFF.

Time to drop storages, during shrink, summed. Basically zero.

Time to drop storages, during clean, summed. With recycler OFF, this is almost always zero. With recycler ON, this periodically takes 80-120 ms. Interesting...

Looking at clean and shrink overall, the total times for each appear to be mostly the same.

brooksprumo · 2024-03-06T21:24:06Z

accounts-db/src/ancient_append_vecs.rs

+        // NOTE: The recycler has been removed.  Creating this many extra storages is no longer
+        // necessary, but also does no harm either.
+        const MAX_RECYCLE_STORES: usize = 1000;


The MAX_RECYCLE_STORES is now only used in this test. In an effort to create a small PR, I have not updated the test here.

brooksprumo · 2024-03-06T21:25:04Z

runtime/src/snapshot_minimizer.rs

-        let (_, drop_or_recycle_stores_measure) = measure!(
-            self.accounts_db()
-                .drop_or_recycle_stores(dead_storages, &self.accounts_db().shrink_stats),
-            "drop or recycle stores"
-        );
-        info!("{drop_or_recycle_stores_measure}");


The drop_or_recycle_stores() function has been removed, so we call drop explicitly now.

brooksprumo · 2024-03-06T21:33:26Z

accounts-db/src/accounts_db.rs

            );
        }

-        self.drop_or_recycle_stores(dead_storages, stats);


This function was removed, so we call drop directly now.

brooksprumo · 2024-03-06T21:35:04Z

accounts-db/src/accounts_db.rs

-    pub fn drop_or_recycle_stores(
-        &self,
-        dead_storages: Vec<Arc<AccountStorageEntry>>,
-        stats: &ShrinkStats,
-    ) {
-        let mut recycle_stores_write_elapsed = Measure::start("recycle_stores_write_time");
-        let mut recycle_stores = self.recycle_stores.write().unwrap();
-        recycle_stores_write_elapsed.stop();
-
-        let mut drop_storage_entries_elapsed = Measure::start("drop_storage_entries_elapsed");
-        if recycle_stores.entry_count() < MAX_RECYCLE_STORES {
-            recycle_stores.add_entries(dead_storages);
-            drop(recycle_stores);
-        } else {
-            self.stats
-                .dropped_stores
-                .fetch_add(dead_storages.len() as u64, Ordering::Relaxed);
-            drop(recycle_stores);
-            drop(dead_storages);
-        }
-        drop_storage_entries_elapsed.stop();
-        stats
-            .drop_storage_entries_elapsed
-            .fetch_add(drop_storage_entries_elapsed.as_us(), Ordering::Relaxed);
-        stats
-            .recycle_stores_write_elapsed
-            .fetch_add(recycle_stores_write_elapsed.as_us(), Ordering::Relaxed);
-    }


When not recycling, this function boils down to:

drop the storages

update stats

We inline this into the one real caller above.

Yah. Probably we can save 200us.

brooksprumo · 2024-03-06T21:36:06Z

accounts-db/src/accounts_db.rs

            .safety_checks_elapsed
            .fetch_add(safety_checks_elapsed.as_us(), Ordering::Relaxed);

-        let mut total_removed_storage_entries = 0;


This is a duplicate of num_stored_slots_removed, so we just use that one instead for the stats.

brooksprumo · 2024-03-06T21:37:04Z

accounts-db/src/accounts_db.rs

+        self.stats
+            .dropped_stores
+            .fetch_add(num_stored_slots_removed as u64, Ordering::Relaxed);


This stat update used to live in recycle_slot_stores() (which was removed).

codecov-commenter · 2024-03-06T23:02:41Z

Codecov Report

Attention: Patch coverage is 90.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 81.8%. Comparing base (c161351) to head (e7fbb6c).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master     #118     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%     
=========================================
  Files         838      838             
  Lines      225947   225700    -247     
=========================================
- Hits       184971   184717    -254     
- Misses      40976    40983      +7

jeffwashington · 2024-03-07T16:06:00Z

First pass review looks good to me. I'm going to use a finer tooth comb and go over it again and think about this for a while.

HaoranYi · 2024-03-07T19:21:20Z

Overall looks good.

From the stats on mainnet, it looks like the recycle store is almost always full with 1K entry. I guess this means that not many of the entries in the store got recycled?

Do we have any other metric that directly count how many stores are recycled?

jeffwashington

lgtm
super excited to reduce complexity and get rid of this whole concept. It always attacks suspicion and will simplify the model of how to think about storages.

brooksprumo · 2024-03-07T21:14:32Z

Do we have any other metric that directly count how many stores are recycled?

Here's metrics from a 15 minute interval for all of the canaries running on mnb:

The spikes are likely for each run of flush/clean/shrink, every 100 or so slots.

HaoranYi · 2024-03-07T22:21:14Z

Do we have any other metric that directly count how many stores are recycled?

Here's metrics from a 15 minute interval for all of the canaries running on mnb:

The spikes are likely for each run of flush/clean/shrink, every 100 or so slots.

Ok. Looks like we don't recycle much, ie 10%.

HaoranYi · 2024-03-07T22:23:30Z

accounts-db/src/accounts_db.rs

-        let mut drop_storage_entries_elapsed = Measure::start("drop_storage_entries_elapsed");
        // Backing mmaps for removed storages entries explicitly dropped here outside
        // of any locks
+        let mut drop_storage_entries_elapsed = Measure::start("drop_storage_entries_elapsed");


nit: measure!(...) as we did for other places.

Yep, I could do that. I was trying to keep any extraneous changes to a minimum. Since it's not a correctness issue, I'd prefer to not tackle that change here in this PR though.

HaoranYi

lgtm.

brooksprumo · 2024-03-07T22:30:42Z

@sakridge Requesting your review to ensure there's not any other gotchas/corner cases I've missed. Since you added the recycler, I figured your input would be valuable. I'm happy to look up any additional metrics too.

sakridge · 2024-03-10T22:09:47Z

Now that we have the accounts write cache, only roots are written to storages. IOW, we don't have storages for slots on a minority fork that got discarded.

Ideally all slots are roots or a very high percentage, so I'm not sure how having the write cache only write roots should change much in terms of file creation assuming we still create new files for each slot.

I think we do write the whole slot to a single file now instead of potentially multiple files as before, so maybe that would reduce the average number of files created.

brooksprumo · 2024-03-11T12:41:13Z

[..] I'm not sure how having the write cache only write roots should change much in terms of file creation assuming we still create new files for each slot.

My understanding was that before the write cache was added, append vecs were created for every slot, before rooting. Yes, most would get rooted, but not all. So some append vecs would get dropped sooner because their slot was pruned due to being on a minority fork.

I think we do write the whole slot to a single file now instead of potentially multiple files as before, so maybe that would reduce the average number of files created.

Yes, now we write-all instead of append. If there is only a single account path, then I think the number of storages would be the same. With multiple account path, then yes, before there could be multiple storages per slot. Combined with the part above, dropped slots on a minority fork could drop more storages if there were many account paths.

brooksprumo · 2024-03-11T19:19:22Z

I've updated the PR's description and added a bunch of metrics under the "More Metrics" details dropdown.

sakridge

Ok, from the metrics seems to be not helping much given the save in complexity and the drop overhead. Thanks adding those additional ones.

brooksprumo · 2024-03-11T19:38:25Z

Ok, from the metrics seems to be not helping much given the save in complexity and the drop overhead.

I'll keep monitoring these metrics going forward to see if there are any regressions.

only reroute if relayer connected (anza-xyz#123) feat: add client tls config (anza-xyz#121) remove extra val (anza-xyz#129) fix clippy (anza-xyz#130) copy all binaries to docker-output (anza-xyz#131) Ledger tool halts at slot passed to create-snapshot (anza-xyz#118) update program submodule (anza-xyz#133) quick fix for tips and clearing old bundles (anza-xyz#135) update submodule to new program (anza-xyz#136) Improve stake-meta-generator usability (anza-xyz#134) pinning submodule head (anza-xyz#140) Use BundleAccountLocker when handling tip txs (anza-xyz#147) Add metrics for relayer + block engine proxy (anza-xyz#149) Build claim-mev in docker (anza-xyz#141) Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154) update submodule + dev files (anza-xyz#158) Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159) update jito-programs submodule (anza-xyz#160) Separate MEV tip related workflow (anza-xyz#161) Add block builder fee protos (anza-xyz#162) fix jito programs (anza-xyz#163) update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164) Pipe through block builder fee (anza-xyz#167) pull in new snapshot code (anza-xyz#171) block builder bug (anza-xyz#172) Pull in new slack autosnapshot submodule (anza-xyz#174) sort stake meta json and use int math (anza-xyz#176) add accountsdb conn submod (anza-xyz#169) Update tip distribution parameters (anza-xyz#177) new submodules (anza-xyz#180) Add buildkite link for jito CI (anza-xyz#183) Fixed broken links to repositories (anza-xyz#184) Changed from ssh to https transfer for clone Seg/update submods (anza-xyz#187) fix tests (anza-xyz#190) rm geyser submod (anza-xyz#192) rm dangling geyser references (anza-xyz#193) fix syntax err (anza-xyz#195) use deterministic req ids in batch calls (anza-xyz#199) update jito-programs revert cargo update Cargo lock update with path fix fix cargo update autosnapshot with block lookback (anza-xyz#201) [JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203) Add logging for sol balance desired (anza-xyz#205) * add logging * add logging * update msg * tweak vars update submodule (anza-xyz#204) use efficient data structures when calling batch_simulate_bundles (anza-xyz#206) [JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209) add config to simulate on top of working bank (anza-xyz#211) rm frozen bank check simulate_bundle rpc bugfixes (anza-xyz#214) rm frozen bank check in simulate_bundle rpc method [JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210) * add files * switch to include bump update submodule (anza-xyz#217) add amount filter (anza-xyz#218) update autosnapshot (anza-xyz#222) Print TX error in Bundles (anza-xyz#223) add new args to support single relayer and block-engine endpoints (anza-xyz#224) point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228) fix clippy errors (anza-xyz#230) fix validator start scripts (anza-xyz#232) Point README to gitbook (anza-xyz#237) use packaged cargo bin to build (anza-xyz#239) Add validator identity pubkey to StakeMeta (anza-xyz#226) The vote account associated with a validator is not a permanent link, so log the validator identity as well. bugfix: conditionally compile with debug flags (anza-xyz#240) Seg/tip distributor master (anza-xyz#242) * validate tree nodes * fix unit tests * pr feedback * bump jito-programs submod Simplify bootstrapping (anza-xyz#241) * startup without precompile * update spacing * use release mode * spacing fix validation rm validation skip Account for block builder fee when generating excess tip balance (anza-xyz#247) Improve docker caching delay constructing claim mev txs (anza-xyz#253) fix stake meta tests from bb fee (anza-xyz#254) fix tests Buffer bundles that exceed cost model (anza-xyz#225) * buffer bundles that exceed cost model clear qos failed bundles buffer if not leader soon (anza-xyz#260) update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265) fix simulate_bundle client and better error handling (anza-xyz#267) update submod (anza-xyz#272) Preallocate Bundle Cost (anza-xyz#238) fix Dockerfile (anza-xyz#278) Fix Tests (anza-xyz#279) Fix Tests (anza-xyz#281) * fix tests update jito-programs submod (anza-xyz#282) add reclaim rent workflow (anza-xyz#283) update jito-programs submod fix clippy errs rm wrong assertion and swap out file write fn call (anza-xyz#292) Remove security.md (anza-xyz#293) demote frequent relayer_stage-stream_error to warn (anza-xyz#275) account for case where TDA exists but not allocated (anza-xyz#295) implement better retries for tip-distributor workflows (anza-xyz#297) limit number of concurrent rpc calls (anza-xyz#298) Discard Empty Packet Batches (anza-xyz#299) Identity Hotswap (anza-xyz#290) small fixes (anza-xyz#305) Set backend config from admin rpc (anza-xyz#304) Admin Shred Receiver Change (anza-xyz#306) Seg/rm bundle UUID (anza-xyz#309) Fix github workflow to recursively clone (anza-xyz#327) Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341) Use cluster info functions for tpu (anza-xyz#345) Use git rev-parse for git sha Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374) Updates bootstrap and start scripts needed for local dev. (anza-xyz#384) Remove Deprecated Cli Args (anza-xyz#387) Master Rebase improve simulate_bundle errors and response (anza-xyz#404) derive Clone on accountoverrides (anza-xyz#416) Add upsert to AccountOverrides (anza-xyz#419) update jito-programs (anza-xyz#430) [JIT-1661] Faster Autosnapshot (anza-xyz#436) Reverts simulate_transaction result calls to upstream (anza-xyz#446) Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449) first pass at wiring up jito-plugin (anza-xyz#428) [JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489) [JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491) [JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490) Add Blockhash Metrics to Bundle Committer (anza-xyz#500) add priority fee ix to mev-claim (anza-xyz#520) Update Autosnapshot (anza-xyz#548) Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582) Update CI (anza-xyz#584) - Add recursive submodule checkouts. - Re-add solana-secondary step Add more release fixes (anza-xyz#585) Fix more release urls (anza-xyz#588) [JIT-1812] Fix blocking mutexs (anza-xyz#495) [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587) Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success. Fix periodic rebase anza-xyz#594 Fixes the following bugs in the periodic rebase: Sends multiple messages on failure instead of one Cancels entire job if one branch fails Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597) Sleep longer waiting for buildkite to start (anza-xyz#598) correctly initialize account overrides (anza-xyz#595) Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603) Add fast replay branch to daily rebase (anza-xyz#607) take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615) update jito-programs submodule Add 2.0 to daily rebase (anza-xyz#626) Export agave binaries during docker build (anza-xyz#627) Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611) Publish releases to S3 and GCS (anza-xyz#633) Rebase from different repos (anza-xyz#637) Point SECURITY.md to immunefi (anza-xyz#671) Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683) Separate out broadcast + retransmit shredstream (anza-xyz#703) Add packet flag for staked node (anza-xyz#705) Add auto-rebase to v2.1 (anza-xyz#739) Fix release github (anza-xyz#745) Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753) Add precompile checks in BundleStage (anza-xyz#787) Add auto-rebase to v2.2 (anza-xyz#818) Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860) Remove unwrap from authentication (anza-xyz#861) Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873) BP anza-xyz#885: Add libclang to Dockerfile (anza-xyz#886) Remove the tip distributor code (anza-xyz#888) Rebase: Update anchor to not use deprecated crates

only reroute if relayer connected (anza-xyz#123) feat: add client tls config (anza-xyz#121) remove extra val (anza-xyz#129) fix clippy (anza-xyz#130) copy all binaries to docker-output (anza-xyz#131) Ledger tool halts at slot passed to create-snapshot (anza-xyz#118) update program submodule (anza-xyz#133) quick fix for tips and clearing old bundles (anza-xyz#135) update submodule to new program (anza-xyz#136) Improve stake-meta-generator usability (anza-xyz#134) pinning submodule head (anza-xyz#140) Use BundleAccountLocker when handling tip txs (anza-xyz#147) Add metrics for relayer + block engine proxy (anza-xyz#149) Build claim-mev in docker (anza-xyz#141) Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154) update submodule + dev files (anza-xyz#158) Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159) update jito-programs submodule (anza-xyz#160) Separate MEV tip related workflow (anza-xyz#161) Add block builder fee protos (anza-xyz#162) fix jito programs (anza-xyz#163) update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164) Pipe through block builder fee (anza-xyz#167) pull in new snapshot code (anza-xyz#171) block builder bug (anza-xyz#172) Pull in new slack autosnapshot submodule (anza-xyz#174) sort stake meta json and use int math (anza-xyz#176) add accountsdb conn submod (anza-xyz#169) Update tip distribution parameters (anza-xyz#177) new submodules (anza-xyz#180) Add buildkite link for jito CI (anza-xyz#183) Fixed broken links to repositories (anza-xyz#184) Changed from ssh to https transfer for clone Seg/update submods (anza-xyz#187) fix tests (anza-xyz#190) rm geyser submod (anza-xyz#192) rm dangling geyser references (anza-xyz#193) fix syntax err (anza-xyz#195) use deterministic req ids in batch calls (anza-xyz#199) update jito-programs revert cargo update Cargo lock update with path fix fix cargo update autosnapshot with block lookback (anza-xyz#201) [JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203) Add logging for sol balance desired (anza-xyz#205) * add logging * add logging * update msg * tweak vars update submodule (anza-xyz#204) use efficient data structures when calling batch_simulate_bundles (anza-xyz#206) [JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209) add config to simulate on top of working bank (anza-xyz#211) rm frozen bank check simulate_bundle rpc bugfixes (anza-xyz#214) rm frozen bank check in simulate_bundle rpc method [JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210) * add files * switch to include bump update submodule (anza-xyz#217) add amount filter (anza-xyz#218) update autosnapshot (anza-xyz#222) Print TX error in Bundles (anza-xyz#223) add new args to support single relayer and block-engine endpoints (anza-xyz#224) point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228) fix clippy errors (anza-xyz#230) fix validator start scripts (anza-xyz#232) Point README to gitbook (anza-xyz#237) use packaged cargo bin to build (anza-xyz#239) Add validator identity pubkey to StakeMeta (anza-xyz#226) The vote account associated with a validator is not a permanent link, so log the validator identity as well. bugfix: conditionally compile with debug flags (anza-xyz#240) Seg/tip distributor master (anza-xyz#242) * validate tree nodes * fix unit tests * pr feedback * bump jito-programs submod Simplify bootstrapping (anza-xyz#241) * startup without precompile * update spacing * use release mode * spacing fix validation rm validation skip Account for block builder fee when generating excess tip balance (anza-xyz#247) Improve docker caching delay constructing claim mev txs (anza-xyz#253) fix stake meta tests from bb fee (anza-xyz#254) fix tests Buffer bundles that exceed cost model (anza-xyz#225) * buffer bundles that exceed cost model clear qos failed bundles buffer if not leader soon (anza-xyz#260) update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265) fix simulate_bundle client and better error handling (anza-xyz#267) update submod (anza-xyz#272) Preallocate Bundle Cost (anza-xyz#238) fix Dockerfile (anza-xyz#278) Fix Tests (anza-xyz#279) Fix Tests (anza-xyz#281) * fix tests update jito-programs submod (anza-xyz#282) add reclaim rent workflow (anza-xyz#283) update jito-programs submod fix clippy errs rm wrong assertion and swap out file write fn call (anza-xyz#292) Remove security.md (anza-xyz#293) demote frequent relayer_stage-stream_error to warn (anza-xyz#275) account for case where TDA exists but not allocated (anza-xyz#295) implement better retries for tip-distributor workflows (anza-xyz#297) limit number of concurrent rpc calls (anza-xyz#298) Discard Empty Packet Batches (anza-xyz#299) Identity Hotswap (anza-xyz#290) small fixes (anza-xyz#305) Set backend config from admin rpc (anza-xyz#304) Admin Shred Receiver Change (anza-xyz#306) Seg/rm bundle UUID (anza-xyz#309) Fix github workflow to recursively clone (anza-xyz#327) Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341) Use cluster info functions for tpu (anza-xyz#345) Use git rev-parse for git sha Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374) Updates bootstrap and start scripts needed for local dev. (anza-xyz#384) Remove Deprecated Cli Args (anza-xyz#387) Master Rebase improve simulate_bundle errors and response (anza-xyz#404) derive Clone on accountoverrides (anza-xyz#416) Add upsert to AccountOverrides (anza-xyz#419) update jito-programs (anza-xyz#430) [JIT-1661] Faster Autosnapshot (anza-xyz#436) Reverts simulate_transaction result calls to upstream (anza-xyz#446) Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449) first pass at wiring up jito-plugin (anza-xyz#428) [JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489) [JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491) [JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490) Add Blockhash Metrics to Bundle Committer (anza-xyz#500) add priority fee ix to mev-claim (anza-xyz#520) Update Autosnapshot (anza-xyz#548) Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582) Update CI (anza-xyz#584) - Add recursive submodule checkouts. - Re-add solana-secondary step Add more release fixes (anza-xyz#585) Fix more release urls (anza-xyz#588) [JIT-1812] Fix blocking mutexs (anza-xyz#495) [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587) Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success. Fix periodic rebase anza-xyz#594 Fixes the following bugs in the periodic rebase: Sends multiple messages on failure instead of one Cancels entire job if one branch fails Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597) Sleep longer waiting for buildkite to start (anza-xyz#598) correctly initialize account overrides (anza-xyz#595) Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603) Add fast replay branch to daily rebase (anza-xyz#607) take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615) update jito-programs submodule Add 2.0 to daily rebase (anza-xyz#626) Export agave binaries during docker build (anza-xyz#627) Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611) Publish releases to S3 and GCS (anza-xyz#633) Rebase from different repos (anza-xyz#637) Point SECURITY.md to immunefi (anza-xyz#671) Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683) Separate out broadcast + retransmit shredstream (anza-xyz#703) Add packet flag for staked node (anza-xyz#705) Add auto-rebase to v2.1 (anza-xyz#739) Fix release github (anza-xyz#745) Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753) Add precompile checks in BundleStage (anza-xyz#787) Add auto-rebase to v2.2 (anza-xyz#818) Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860) Remove unwrap from authentication (anza-xyz#861) Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873) Add libclang to Dockerfile (anza-xyz#885) Remove the tip distributor code (anza-xyz#888) Rebase: Update anchor to not use deprecated crates Add TLS webpki roots back in (anza-xyz#933) Remove trusted relayer packets (anza-xyz#952) Fix shred retransmit (anza-xyz#954) [v3.0] Automatically use optimal Block Engine region (anza-xyz#983) (Backport: anza-xyz#995) Disable autoconfig (anza-xyz#996)

only reroute if relayer connected (anza-xyz#123) feat: add client tls config (anza-xyz#121) remove extra val (anza-xyz#129) fix clippy (anza-xyz#130) copy all binaries to docker-output (anza-xyz#131) Ledger tool halts at slot passed to create-snapshot (anza-xyz#118) update program submodule (anza-xyz#133) quick fix for tips and clearing old bundles (anza-xyz#135) update submodule to new program (anza-xyz#136) Improve stake-meta-generator usability (anza-xyz#134) pinning submodule head (anza-xyz#140) Use BundleAccountLocker when handling tip txs (anza-xyz#147) Add metrics for relayer + block engine proxy (anza-xyz#149) Build claim-mev in docker (anza-xyz#141) Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154) update submodule + dev files (anza-xyz#158) Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159) update jito-programs submodule (anza-xyz#160) Separate MEV tip related workflow (anza-xyz#161) Add block builder fee protos (anza-xyz#162) fix jito programs (anza-xyz#163) update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164) Pipe through block builder fee (anza-xyz#167) pull in new snapshot code (anza-xyz#171) block builder bug (anza-xyz#172) Pull in new slack autosnapshot submodule (anza-xyz#174) sort stake meta json and use int math (anza-xyz#176) add accountsdb conn submod (anza-xyz#169) Update tip distribution parameters (anza-xyz#177) new submodules (anza-xyz#180) Add buildkite link for jito CI (anza-xyz#183) Fixed broken links to repositories (anza-xyz#184) Changed from ssh to https transfer for clone Seg/update submods (anza-xyz#187) fix tests (anza-xyz#190) rm geyser submod (anza-xyz#192) rm dangling geyser references (anza-xyz#193) fix syntax err (anza-xyz#195) use deterministic req ids in batch calls (anza-xyz#199) update jito-programs revert cargo update Cargo lock update with path fix fix cargo update autosnapshot with block lookback (anza-xyz#201) [JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203) Add logging for sol balance desired (anza-xyz#205) * add logging * add logging * update msg * tweak vars update submodule (anza-xyz#204) use efficient data structures when calling batch_simulate_bundles (anza-xyz#206) [JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209) add config to simulate on top of working bank (anza-xyz#211) rm frozen bank check simulate_bundle rpc bugfixes (anza-xyz#214) rm frozen bank check in simulate_bundle rpc method [JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210) * add files * switch to include bump update submodule (anza-xyz#217) add amount filter (anza-xyz#218) update autosnapshot (anza-xyz#222) Print TX error in Bundles (anza-xyz#223) add new args to support single relayer and block-engine endpoints (anza-xyz#224) point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228) fix clippy errors (anza-xyz#230) fix validator start scripts (anza-xyz#232) Point README to gitbook (anza-xyz#237) use packaged cargo bin to build (anza-xyz#239) Add validator identity pubkey to StakeMeta (anza-xyz#226) The vote account associated with a validator is not a permanent link, so log the validator identity as well. bugfix: conditionally compile with debug flags (anza-xyz#240) Seg/tip distributor master (anza-xyz#242) * validate tree nodes * fix unit tests * pr feedback * bump jito-programs submod Simplify bootstrapping (anza-xyz#241) * startup without precompile * update spacing * use release mode * spacing fix validation rm validation skip Account for block builder fee when generating excess tip balance (anza-xyz#247) Improve docker caching delay constructing claim mev txs (anza-xyz#253) fix stake meta tests from bb fee (anza-xyz#254) fix tests Buffer bundles that exceed cost model (anza-xyz#225) * buffer bundles that exceed cost model clear qos failed bundles buffer if not leader soon (anza-xyz#260) update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265) fix simulate_bundle client and better error handling (anza-xyz#267) update submod (anza-xyz#272) Preallocate Bundle Cost (anza-xyz#238) fix Dockerfile (anza-xyz#278) Fix Tests (anza-xyz#279) Fix Tests (anza-xyz#281) * fix tests update jito-programs submod (anza-xyz#282) add reclaim rent workflow (anza-xyz#283) update jito-programs submod fix clippy errs rm wrong assertion and swap out file write fn call (anza-xyz#292) Remove security.md (anza-xyz#293) demote frequent relayer_stage-stream_error to warn (anza-xyz#275) account for case where TDA exists but not allocated (anza-xyz#295) implement better retries for tip-distributor workflows (anza-xyz#297) limit number of concurrent rpc calls (anza-xyz#298) Discard Empty Packet Batches (anza-xyz#299) Identity Hotswap (anza-xyz#290) small fixes (anza-xyz#305) Set backend config from admin rpc (anza-xyz#304) Admin Shred Receiver Change (anza-xyz#306) Seg/rm bundle UUID (anza-xyz#309) Fix github workflow to recursively clone (anza-xyz#327) Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341) Use cluster info functions for tpu (anza-xyz#345) Use git rev-parse for git sha Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374) Updates bootstrap and start scripts needed for local dev. (anza-xyz#384) Remove Deprecated Cli Args (anza-xyz#387) Master Rebase improve simulate_bundle errors and response (anza-xyz#404) derive Clone on accountoverrides (anza-xyz#416) Add upsert to AccountOverrides (anza-xyz#419) update jito-programs (anza-xyz#430) [JIT-1661] Faster Autosnapshot (anza-xyz#436) Reverts simulate_transaction result calls to upstream (anza-xyz#446) Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449) first pass at wiring up jito-plugin (anza-xyz#428) [JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489) [JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491) [JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490) Add Blockhash Metrics to Bundle Committer (anza-xyz#500) add priority fee ix to mev-claim (anza-xyz#520) Update Autosnapshot (anza-xyz#548) Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582) Update CI (anza-xyz#584) - Add recursive submodule checkouts. - Re-add solana-secondary step Add more release fixes (anza-xyz#585) Fix more release urls (anza-xyz#588) [JIT-1812] Fix blocking mutexs (anza-xyz#495) [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587) Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success. Fix periodic rebase anza-xyz#594 Fixes the following bugs in the periodic rebase: Sends multiple messages on failure instead of one Cancels entire job if one branch fails Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597) Sleep longer waiting for buildkite to start (anza-xyz#598) correctly initialize account overrides (anza-xyz#595) Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603) Add fast replay branch to daily rebase (anza-xyz#607) take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615) update jito-programs submodule Add 2.0 to daily rebase (anza-xyz#626) Export agave binaries during docker build (anza-xyz#627) Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611) Publish releases to S3 and GCS (anza-xyz#633) Rebase from different repos (anza-xyz#637) Point SECURITY.md to immunefi (anza-xyz#671) Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683) Separate out broadcast + retransmit shredstream (anza-xyz#703) Add packet flag for staked node (anza-xyz#705) Add auto-rebase to v2.1 (anza-xyz#739) Fix release github (anza-xyz#745) Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753) Add precompile checks in BundleStage (anza-xyz#787) Add auto-rebase to v2.2 (anza-xyz#818) Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860) Remove unwrap from authentication (anza-xyz#861) Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873) Add libclang to Dockerfile (anza-xyz#885) Remove the tip distributor code (anza-xyz#888) Rebase: Update anchor to not use deprecated crates Add TLS webpki roots back in (anza-xyz#933) Remove trusted relayer packets (anza-xyz#952) Fix shred retransmit (anza-xyz#954) Add daily v3.0 rebase (anza-xyz#972) [Master] Automatically use optimal Block Engine region (anza-xyz#974) Disable autoconfig (anza-xyz#995) Make shredstream optional (anza-xyz#997)

brooksprumo self-assigned this Mar 6, 2024

brooksprumo force-pushed the recycle/remove branch from 3857c26 to 2f6bd0c Compare March 6, 2024 21:27

Removes the storage recycler

e7fbb6c

brooksprumo force-pushed the recycle/remove branch from 2f6bd0c to e7fbb6c Compare March 6, 2024 21:30

brooksprumo commented Mar 6, 2024

View reviewed changes

This was referenced Mar 6, 2024

Removes atomic-ness from AccountStorageEntry id and slot fields #119

Merged

Removes holding storages in AccountsHashVerifier for fastboot #120

Merged

brooksprumo marked this pull request as ready for review March 6, 2024 22:15

brooksprumo requested review from HaoranYi and jeffwashington March 7, 2024 14:11

jeffwashington approved these changes Mar 7, 2024

View reviewed changes

brooksprumo requested a review from sakridge March 7, 2024 22:19

HaoranYi reviewed Mar 7, 2024

View reviewed changes

HaoranYi approved these changes Mar 7, 2024

View reviewed changes

sakridge approved these changes Mar 11, 2024

View reviewed changes

brooksprumo merged commit 5c1df15 into anza-xyz:master Mar 11, 2024

brooksprumo deleted the recycle/remove branch March 11, 2024 19:38

This was referenced Mar 21, 2024

Fastboot fails if the node crashes while archiving a full snapshot solana-labs/solana#35367

Closed

Removes AccountsFile::is_recyclable() #359

Merged

codebender828 pushed a commit to codebender828/agave that referenced this pull request Oct 3, 2024

Removes the storage recycler (anza-xyz#118)

cfe6868

OliverNChalk pushed a commit to OliverNChalk/agave that referenced this pull request Nov 11, 2025

public cluster safety check (anza-xyz#118)

699d749

Removes the storage recycler #118

Removes the storage recycler #118

Uh oh!

Conversation

brooksprumo commented Mar 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Summary of Changes

More information

Footnotes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Mar 6, 2024

Codecov Report

Uh oh!

jeffwashington commented Mar 7, 2024

Uh oh!

HaoranYi commented Mar 7, 2024

Uh oh!

jeffwashington left a comment

Choose a reason for hiding this comment

Uh oh!

brooksprumo commented Mar 7, 2024

Uh oh!

HaoranYi commented Mar 7, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HaoranYi left a comment

Choose a reason for hiding this comment

Uh oh!

brooksprumo commented Mar 7, 2024

Uh oh!

sakridge commented Mar 10, 2024

Uh oh!

brooksprumo commented Mar 11, 2024

Uh oh!

brooksprumo commented Mar 11, 2024

Uh oh!

sakridge left a comment

Choose a reason for hiding this comment

Uh oh!

brooksprumo commented Mar 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

brooksprumo commented Mar 6, 2024 •

edited

Loading