Exploration vs Exploitation
The trade-off between showing users what the system currently believes is best (exploitation) and deliberately showing something uncertain to gather signal (exploration). In search ranking it is the standard cure for the feedback loop where a model trained on its own click-through data reinforces Position Bias and Presentation Bias: without exploration, items the ranker never surfaces never get a chance to prove themselves, and the model can slowly degrade.
Exploration segments in ranking
A practical implementation described in Roman Grebennikov - Personalizing Search Results in Real-Time (Findify, MICES 2019): route a small slice of traffic (~1% or less) into an exploration segment where the first page of results is shuffled — still relevant results, but in random order. Averaged over many impressions of the same query, position effects cancel out, revealing what genuinely attracts clicks. Only this randomized segment feeds Learning to Rank training; the remaining exploitation segment serves the best-known ranking. At Findify this stopped previously unexplained model degradation.
Trade-offs:
- Randomization has a real business cost — full randomization measurably trashes conversion, so the segment must stay small.
- For low-traffic merchants/sites, a tiny exploration slice takes impractically long to accumulate training data — one motivation for cross-site generic models.
Beyond traffic splits
- Bandits — contextual bandits balance explore/exploit continuously per decision (see Patterns for Personalization).
- Active exploration of underexposed items — randomly promote long-tail items to create exposure (see Presentation Bias).
- Facet/UI selection — you can’t measure demand for facets you never show (see Facets, But Which Ones).
- Interleaving — an alternative low-risk way to gather comparative feedback online.
Related Concepts
- Position Bias · Presentation Bias
- Learning to Rank · Implicit Judgments · Click Signals · Click Models
- A-B Testing for Search · Interleaving