Course map 14 lessons

LESSON 07 / 14 · Learn to retrieve

Reciprocal Rank Fusion: calculate and inspect every vote

Fuse two ranked lists by hand, handle duplicates and ties, and understand why an RRF score is not confidence.

5 min readAuthor: MESA contributorsPublished: Download Markdown ↓
BY THE END, YOU CAN
  • Compute RRF for three candidates
  • Understand the smoothing constant and lane weights
  • Specify deduplication, missing ranks and tie policy

Before this lesson: Hybrid retrieval: candidates, fusion and reranking

Combine agreement without comparing raw scores#

Two retrievers disagree about the Atlas archive. Lexical ranks A, B, C; dense ranks B, C, D. You want a method that uses each list’s order without pretending their raw scores share units. Reciprocal Rank Fusion assigns a decreasing contribution to each rank and sums the contributions for the same candidate. It is a simple baseline with explicit assumptions.

Cormack, Clarke and Büttcher evaluated this method on specific retrieval experiments. Their results motivate testing RRF; they do not guarantee an improvement for your task. The original method uses unweighted votes. Weighted RRF is an extension in which each lane’s contribution is multiplied by a declared weight. Original paper.

Use one-based ranks#

Worked example
RRF(d) = sum over lanes containing d: w_lane / (k + rank_lane(d))
rank starts at 1; missing candidate contributes 0.
Example below: k = 60, every weight = 1.
CandidateLexical rankDense rankRRF contribution sum
A1absent / yok1/61 = 0.016393
B211/62 + 1/61 = 0.032522
C321/63 + 1/62 = 0.032002
Dabsent / yok31/63 = 0.015873

The fused order is B, C, A, D. Agreement across the lists lifts B and C above A, despite A’s first position in one lane. A candidate missing from a truncated list receives zero from that lane, not a vote for rank infinity or rank zero. Keep candidate cutoffs in your experiment record because truncation changes the votes.

What changes when k changes?#

With k=60, first place contributes 1/61 and second place 1/62, a small difference. With k=1, they contribute 1/2 and 1/3, a much larger difference. Smaller k emphasizes top positions; larger k flattens contributions across the retrieved ranks. This k is a smoothing constant, not the number of results returned. Choose it with held-out evidence, not because 60 is universal.

  1. Lexical · A,B,C
  2. Dense · B,C,D
  3. Votes · 1/(k+rank)
  4. Sum by ID · one per lane
  5. Sort · B,C,A,D
The result is a rank signal, not a calibrated probability.

Specify the edge cases before deployment#

Deduplicate a candidate within each lane before assigning its vote. Otherwise a backend returning the same ID twice changes the method. Decide what the ID represents and keep it consistent across lanes. If one lane returns document IDs and another assertion IDs, reconcile them into the same unit before fusion; matching strings alone is not a sound identity contract.

For exactly tied fused scores, use a documented stable rule such as canonical ID ordering. For weighted fusion, reject non-finite weights and decide whether zero disables a lane. Giving two copies of the same retriever separate full weights can double-count correlated evidence. More votes do not make an incorrect or unauthorized source acceptable.

Recalculate after a weight change#

Check your understanding

Set the dense lane weight to zero in the table. What order remains, and what happens to D?

Reveal the explanation

A, B, C keep the lexical order. D has zero contribution; whether zero-score candidates are excluded is an explicit output policy. Do not interpret the zero as proof that D is false or irrelevant.

Sources and further reading#

  1. Cormack, Clarke & Büttcher (2009) · Reciprocal Rank Fusion
  2. MESA Core v0.9.3 · Architecture and limitations
Ready for the next idea?

Try the question before continuing. Your progress is saved in this browser when storage is available.

← Back to the learning map
In this lesson