Query: Adds support for weighted RRF in hybrid search

[neildsh]

Weighted RRF support

This change adds support for weighted RRF in hybrid search.

We allow weights to be negative but the negative sign is used to signal that we should sort scores in ascending order for the corresponding component. The final WRRF score is then computed using the absolute value of the weight.

In this approach, the sign of the weight indicates the interpretation of the ranking itself rather than directly affecting the calculated score:
WRRF(d) = ∑ |w_i| × 1/(k + r_i'(d))

Where:

|w_i| is the absolute value of the weight for the i-th component
r_i'(d) is the rank of document d in the i-th component, with a crucial difference:

• If w_i > 0: r_i'(d) is the original rank (descending order, where 1 is best)
• If w_i < 0: r_i'(d) is the inverted rank (ascending order, where higher number is better)

k = 60

Intuition Behind This Modification

This modification addresses a key scenario in information retrieval and ranking fusion: sometimes "lower is better" rather than "higher is better."

• Handling Different Ranking Interpretations: Some systems naturally produce rankings where lower values are better (like error rates, distances, or pricing). The sign becomes a semantic indicator of how to interpret the ranking.
• Unifying Disparate Metrics: This allows you to combine rankings based on completely different metrics (relevance scores, error rates, freshness, etc.) without having to pre-process them into a common format.
• Integration of Minimization Metrics: You can directly incorporate metrics that should be minimized (like latency or error) alongside metrics that should be maximized (like relevance or user engagement).

Type of change

• New feature (non-breaking change which adds functionality)

[sboshra]

[sboshra]

[adityasa]