Adds SampleSuccGauss FailureHandler for UKI

[ilopezgp]

Adds the Sample-successful-gaussian Failure Handler for the Unscented Kalman Inversion.

The handler is applied during the analysis step, by computing the ill-defined means and covariances over the successful particles. In order to preserve the sum of the weighting scheme, the weights at each successful sigma point are scaled up.

Consider a set of sigma points {θ_j}={θ_succ_j} U {θ_fail_j}, from which we have to estimate E[G], Cov(G, G) and Cov(G, θ) for the UI update rules. These would generally be computed through the quadrature rules of UKI (possibly modified following Huang et al 2021).

For the mean of the forward model eval we do:

• E[G] ≈ ∑_succ [(∑W_j) / (∑_succ W_j) * W_j * G(θ_succ_j)] if the center particle did not fail (rescaled quadrature).
• E[G] ≈ ∑_succ G(θ_succ_j) / len({θ_succ_j}) if the center particle failed (arithmetic mean).

The second option is needed because in the modified unscented transform, W_1 = 1 and all other weights are zero. We do similarly for the mean over parameters θ.

For the covariances we scale up the weights of the off-centered sigma points and compute over the successful particles,

• Cov(G, θ) = ∑_succ [(∑_(j>1) Wc_j) / (∑_(succ, j>1) Wc_j) * Wc_j * (G(θ_succ_j) - G(m)) * (θ_succ_j - m)]
• Cov(G, θ) = ∑_succ [(∑_(j>1) Wc_j) / (∑_(succ, j>1) Wc_j) * Wc_j * (θ_succ_j - m) * (θ_succ_j - m)]
• Cov(G, G) = Wc_1 * (G(m) -E[ G]) * (G(m) -E[ G]) + ∑_succ_(j>1) [(∑_(j>1) Wc_j) / (∑_(succ, j>1) Wc_j) * Wc_j * (G(θ_succ_j) - E[G]) * (G(θ_succ_j) - E[G])]

Note that the term Wc_1 * (G(m) -E[ G]) * (G(m) -E[ G]) disappears when using the modified unscented transform because we take E[G]≈G(m). Also, notice that the rescaling is computed over the off-centered sigma points to remain consistent for both Cov(G, θ) and Cov(G, G).

[Zhengyu-Huang]

The improvement looks great! Thanks!

[odunbar]

Looks good. Two points

• can we guarantee that the IgnoreFailures() will pass through and produce NaNs rather than cause errors? (which is what is tested here)
• can we have some warning flags for >50% fails etc? (as in EKI) - I presume this is still important. It may be worth at some point exporting this out of the individual methods.

[ilopezgp]

Looks good. Two points

• can we guarantee that the IgnoreFailures() will pass through and produce NaNs rather than cause errors? (which is what is tested here)
• can we have some warning flags for >50% fails etc? (as in EKI) - I presume this is still important. It may be worth at some point exporting this out of the individual methods.

Thanks! About the points:

• We can guarantee that IgnoreFailures() will pass through and produce NaNs during a single call to update_ensemble!. However, when the new parameters (NaNs) are passed to the forward model, the behavior is controlled by the forward model. For toy models in Julia, it would still pass through, but for more complex code in possibly other languages or involving IO, it will likely break. This is the same behavior we had prior to adding FailureHandlers, and I think at some point we may want to shift to using an actual FailureHandler as a default.
• The 50% flag is used by all algorithms through split_indices_by_success.

[ilopezgp]

bors r+

[bors]

Build succeeded:

• docs-build
• format
• test_success