Fix OOM error in large parentchain syncs #1468
Conversation
…ck numbers into inner function
clangenb
added
A0-core
|
|
||
| println!( | ||
| "[{:?}] Syncing blocks from {} to {}", | ||
| "[{:?}] Syncing blocks from #{} to #{}", |
There was a problem hiding this comment.
It simply struck me that substrate uses a # before the block numbers, so I thought we can do the same for homogenous logs :)
| println!( | ||
| "[{:?}] sync target #{} is not finalized (#{}), wait a sec ...", | ||
| id, sync_target.number, curr_block_number | ||
| ); | ||
| std::thread::sleep(std::time::Duration::from_secs(1)); | ||
| continue |
There was a problem hiding this comment.
This resolves spamming network connections until the parentchain is ready: #1465
…e arg to a borrow
| #[cfg(feature = "teeracle")] | ||
| let teeracle_tokio_handle = tokio_handle.clone(); |
There was a problem hiding this comment.
No longer needed as another function needs a borrow now instead of an owned handle.
| // initialize teeracle interval | ||
| #[cfg(feature = "teeracle")] | ||
| if WorkerModeProvider::worker_mode() == WorkerMode::Teeracle { |
There was a problem hiding this comment.
The flow of the following section was a bit hard to understand because there were non-exclusive if statements (which is why I didn't notice that it doesn't work yet). Now we have three exclusive paths in the match statement, which make it easier to understand. The code redundancy is reduced to some function calls by extracting some code into functions.
| WorkerMode::Teeracle => { | ||
| // ------------------------------------------------------------------------ | ||
| // initialize teeracle interval | ||
| #[cfg(feature = "teeracle")] |
|
Superseded by #1493, which is based on the new master. |
The problem behind the OOM error was that we had synced the whole parentchain before we imported the blocks into the light client. This PR changes the behaviour to trigger the import of the parentchain blocks after each chunk in the
sync_parentchain_and_import_blocksfunction. The behaviour ofsync_parentchainhas not been changed. This is because it is either used by non-primary workers, which have only small syncs, as they receive a light-client-db from the primary worker at startup, or it is used in the teeracle/offchain worker modes, which directly import the parentchain blocks when they are synced. Closes sidechain oom & preserve db #1462.Another minor fix is included that fixes startup error: spam the loop to get parentchain blocks until it has finalized the sync target. #1465