Async transfers to host for multiple salts #4033
Description
Here's an idea on how we can speed up some OpenCL formats when working with multiple salts:
crypt_all is passed a pointer to the current salt in the database. It can also access salt->next->salt for likely next call's salt (except in self-test, benchmark, and "single crack" modes). When salt->next is non-NULL, before starting (edit: or waiting for completion of) transfer of computed hashes from device to host crypt_all may set the next salt and invoke the kernel for that one (with the set of candidate passwords previously transferred to device). crypt_all will also skip kernel invocation for the current salt if it's already been invoked by the previous call and the salt was predicted correctly (e.g., set_salt may reset a flag to indicate occasional misprediction - which should never happen with usual cracking modes). That way, for most salts (for all but the first one in salts list) transfers from device to host will overlap in time with computation of hashes for next salt.
In order to allow for such overlapping, we need two (sets of) output buffers (both on device and on host), pointers to which will be swapped before each kernel invocation. The rest of the format's functions such as cmp_* and get_hash_* will read from the buffer that was used for the current salt (not the next salt).