MOC — Ranking and Retrieval

Map of content covering lexical retrieval, learning to rank, hybrid fusion, multi-stage pipelines, and personalization.

Lexical Retrieval

BM25 and TF-IDF

• Concept: BM25 — the standard for lexical retrieval; k1/b parameters; IDF saturation

• Understanding the BM25 Algorithm — clear formula breakdown with k1/b intuition

• Targeting Broad Queries in Search — Etsy: broad query challenges for BM25

• How Etsy Uses Thermodynamics for Search — entropy-based diversity for broad queries

• Semantic Search Without Embeddings — Doug Turnbull; taxonomy + BM25 as alternative to dense retrieval

Classic Search Problems

• Broad and Ambiguous Search Queries — Tunkelang on disambiguation strategies
• Ecommerce Search UX - 8 Query Types — Baymard’s query taxonomy

Hybrid Search

• Concept: Hybrid Search — sparse + dense combination
• Concept: Reciprocal Rank Fusion — rank-based merging, score-agnostic fusion
• Concept: Relative Score Fusion — score-based merging, magnitude-preserving
• Concept: Score Normalization — the step that makes score fusion viable
• RRF is Not Enough — why naive RRF fails; upstream quality matters; intent routing
• SPLADE for Sparse Vector Search Explained
• Hybrid Search SPLADE Sparse Encoder

MongoDB Hybrid Search series — Erik Hatcher (MongoDB):

• Survey of the Hybrid Search Landscape — techniques ordered by rankability; measure, tune, repeat

• Reciprocal Rank Fusion and Relative Score Fusion — RRF and RSF worked through with full arithmetic

• Hybrid Search Blueprint Series Semantic Boosting — two-phase alternative to list merging

• Wormhole Vectors Beyond Hybrid Search in OpenSearch — Dima Kan; SKG traversal bridging sparse/dense/behavioral spaces

• Hybrid search > sum of its parts? Berlin Buzzwords 2022 — early hybrid search benchmarking

Vespa zero-shot ranking series — Jo Kristian Bergum (Vespa):

• Improving Zero-Shot Ranking with Vespa Hybrid Search — why in-domain scores don’t predict transfer; BEIR and Dense Passage Retriever
• Improving Zero-Shot Ranking with Vespa Hybrid Search - part two — tuned BM25 + distilled ColBERT with distributed min-max normalization; 0.453 → 0.481 on BEIR
• Improving Search Ranking with Few-Shot Prompting of LLMs — Synthetic Query Generation closing the remaining gap on TREC-COVID
• Vespa - Ranking Without Labels on CORD-19 — the series as a single case study
• Multilingual Embedding Model Hybrid Search Reranking — Quynh Nguyen; cross-lingual E5 + RRF + Cohere reranker

Multi-Stage Ranking

• Concept: Retrieval Pipeline — full retrieve → rerank pipeline

• Multi-Stage Ranking — three stages: retrieval → relevance → click ranking

• Cross-Encoders ColBERT and LLM-Based Re-Rankers — when to use each

• Bi-encoder vs Cross-encoder When to Use Which One

• When Reranking Becomes a System Boundary — Ravindra Harige: retrieval defines eligibility, reranking defines order; compensatory reranking; evaluation and ownership splits across pipeline stages

Learning to Rank

• Concept: Learning to Rank — LambdaMART, RankNet, neural rerankers
• How LambdaMART Works — Turnbull explains lambda gradients and MART
• Machine Learning-Powered Search Ranking of Airbnb Experiences — 4-stage ML progression
• Learning to Rank for Flight Itinerary Search — Skyscanner LTR case study
• Beyond Algorithms - Ranking at Scale at Booking.com — signal selection, leakage, and serving beyond algorithm choice
• Concept: Ranking Signal Selection — choosing the target variable before the model
• Building a Better Search Engine for Semantic Scholar — LightGBM + LambdaRank
• Algorithms: LambdaMART · RankNet · MonoT5 · RankLLaMA · RankGPT · Ranking Objectives
• Implementations: LightGBM · XGBoost · CatBoost · RankLib
• Serving: Metarank — open-source LambdaMART secondary re-ranker; Feature Store · Implicit Judgments · Interleaving
• Learn-to-Rank with OpenSearch and Metarank — Roman Grebennikov; external LTR re-ranking on OpenSearch
• Hybrid Search and Learning-to-Rank with Metarank — Vsevolod Goloviznin; LTR as multi-retriever fusion
• Metarank - Personalized Ranking That Actually Reads Your Clicks — Florian Narr; Metarank repo review
• In-engine LTR: Elasticsearch Learning to Rank — external (Metarank) vs internal, and the two ES modules (OSC plugin vs native 8.12+)
• We’re Bringing Learning to Rank to Elasticsearch — OpenSource Connections; the 2017 RankLib-based plugin
• Learning To Rank (LTR) - Elasticsearch Native — Elastic native LTR; GBDT/XGBoost via eland
• Search using LTR - Elastic Docs — native LTR rescorer / retriever

Personalization

• Concept: Personalization — user embeddings, behavioral signals

• Patterns for Personalization — Eugene Yan: 5 ML patterns with industry examples

• Listing Embeddings in Search Ranking — Airbnb: real-time personalization via embeddings

• From Elasticsearch to Vespa - Rebuilding the Kleinanzeigen Homepage Feed Part 1 — Andre Charton, Kleinanzeigen: WAND-based click profile matching, in-engine user profiles with Vespa, token similarity propagation

Bias and Diversity

• Concept: Position Bias — rank confounder in click signals
• Concept: Impression Bias — coverage confounder; which items were ever shown
• Concept: Diversity Metrics — MMR, α-NDCG, APD, entropy
• Search at Slack — how Slack handles position bias in LTR training
• Beyond Algorithms - Ranking at Scale at Booking.com — impression, position, and user bias named together in a production marketplace ranker
• Diversity Metrics for Recommender Systems — diversity metrics from a RecSys perspective
• Maximal Marginal Relevance for Keyphrase Extraction — MMR applied to keyphrase selection

Related MOCs

• MOC - Search Quality Assurance and Query Understanding
• MOC - Agentic Search and Embeddings
• MOC - Case Studies

Additional Articles — Ranking

• Bayesian BM25 is Cool — BB25: calibrated BM25 probabilities for principled hybrid fusion
• Understanding BERT and Search Relevance — BERT dense vectors for search; storage trade-offs; black box challenge
• E-commerce Search and Recommendation with Vespa — Vespa features for e-commerce search and recs
• Using Approximate Nearest Neighbor Search to Find Similar Products — ANN with metadata filters, HNSW, real-time updates
• Putting Search Ranking in Perspective — Tunkelang on ranking priority stack

New: Search Governance (Ecommerce)

• Why Ecommerce Search Needs Governance and How It Improves Retrieval — Alexander Marquardt et al.; governance layer between query and retrieval; intent routing; Part 1 of series
• Ecommerce Search Governance - Move Faster Not Slower — Alexander Marquardt et al.; zero-deploy operating model; policies as data; Part 2 of series
• Elasticsearch Personalized Search in Ecommerce - Improve Relevance — Alexander Marquardt et al.; purchase history + cohort personalization within governance; Part 6 of series
• Metadata - The 3rd Kind of Retrieval — Doug Turnbull; attribute-based ranking as complement to lexical + semantic

New: BM25 as Probability

• Can BM25 be a Probability — Doug Turnbull; BB25 calibration framework for principled hybrid fusion; gradient descent on alpha/beta