Fix index corruption on reindex #170
Conversation
cheme
left a comment
cheme
left a comment
There was a problem hiding this comment.
👍 didn't thought about this.
| _ => Ok(PlanOutcome::Written), | ||
| let mut outcome = PlanOutcome::Written; | ||
| while let PlanOutcome::NeedReindex = | ||
| tables.index.write_insert_plan(key, address, None, log)? |
There was a problem hiding this comment.
Just to be sure.
Looping is just in case there is some concurrent reindexing leading to a multiple reindex trigger (could happen , but probably rare in practice).
There was a problem hiding this comment.
Yeah, it should never happen realistically. Insertion into an empty index should always be successful. This code had a small bug. An entry in the value table was created twice if reindex was triggered.
| Ok(false) | ||
| } | ||
|
|
||
| fn search_all_indexes<'a>( |
There was a problem hiding this comment.
Makes me think search_all_indexes could be call search_index and search_index something a bit technical, so first idea would be to search all indexes.
But really not needed.
| } else if let Some(table) = reindex.queue.iter().find(|r| r.id == record.table) { | ||
| table.validate_plan(record.index, log)?; | ||
| } else { | ||
| if record.table.index_bits() < tables.index.id.index_bits() { |
There was a problem hiding this comment.
I remember deriving ord the other day when looking at this, just realize it may have been plain wrong (logic to ordering between different index may just be bad).
src/db.rs
Outdated
| f: impl FnMut(IterState) -> bool, | ||
| ) -> Result<()> { | ||
| /// Iterate a column and call a function for each value. Iteration order is unspecified. Note | ||
| /// that for hash columns the key is the hash of the original key. |
There was a problem hiding this comment.
We should add that this iteration skip the commit overlay (iirc this is on dbinner so user should not be calling it, but still would be good to note).
Could also say that this is only for Hash indexed column (btree is unimplemented, since we got iterator, but could actually probably be).
An interrupted reindexing process would result in duplicated index entries on the next attempt. The check for existing entries when reindexing is incorrect, as it checks for full key when only partial key is available.