Click Signals
Definition
Click signals are behavioral data collected from user interactions with search results — clicks, dwell time, reformulations, and conversions. They provide implicit relevance feedback at scale without requiring human annotation.
Types of Click Signals
Click-Through Rate (CTR)
Fraction of queries where a user clicked at least one result.
CTR = clicks / impressions
Useful for: measuring engagement; comparing systems in A/B tests. Limitation: CTR depends heavily on result presentation (position bias).
Dwell Time
How long a user spent on a clicked result before returning.
- Long dwell (>30s): positive signal — user found what they needed
- Short dwell (<5s): negative signal — “pogo-sticking” back to SERP
Zero Clicks
User got results but clicked nothing. Interpretations vary:
- Negative: nothing was relevant
- Positive: query answered directly in snippet/featured snippet (no click needed)
Context matters: a zero-click rate for navigational queries is different from informational queries.
Click Residual
A sophisticated metric: how many “expected” clicks didn’t happen?
Concept: Given a result set of a certain relevance profile, predict how many clicks should occur. The gap between expected and actual clicks = “residual.”
Click Residual = Expected Clicks - Actual Clicks
Positive residual: fewer clicks than expected → results are less relevant than they appear.
Negative residual: more clicks than expected → results are surprisingly engaging.
Position Bias
Users click higher-ranked results more regardless of quality. Must be corrected for when using click signals as relevance proxies:
- Inverse propensity scoring (IPS): weight clicks by inverse probability of being shown at that position
- Counterfactual evaluation: compare clicks under different presentation orderings
Click Data in LTR (Learning to Rank)
Click data is the primary training signal for Learning to Rank models:
- Collect (query, result, click/no-click) data from production
- Apply position debiasing
- Use as implicit relevance labels for LTR training
Key challenge: Clicks are biased toward currently-ranked positions. A result at rank 10 gets few clicks even if it’s the best answer — its click rate doesn’t reflect true relevance.
When Clicks Aren’t Logged: Session Stitching
Not every system records search-to-click events directly. Criteo sees searches and product-page views only as separate events, so for CLEPR it rebuilds pseudo-clicks by joining a search to the page views that follow it in the same session, using timing heuristics. To limit the extra noise, it keeps only keyword–product pairs where the keyword reliably leads to the same product across sessions, with minimum volume and recency. It also counts each pair once regardless of click volume, so popular products don’t dominate the training signal (Introducing CLEPR, our model for semantic understanding).
Click Signals vs. Judgment Lists
| Aspect | Click Signals | Human Judgments |
|---|---|---|
| Scale | Millions/day | Hundreds/day |
| Cost | Free (already collected) | Expensive |
| Bias | Position bias, novelty bias | Annotation bias |
| Quality | Noisy | High (if guidelines good) |
| Freshness | Real-time | Stale quickly |
Related Concepts
-
Search Evaluation — click signals as online evaluation
-
Session-Based Evaluation — clicks as session signals
-
Judgment Lists — offline alternative
-
Implicit Judgments — click signals aggregated into relevance labels for LTR
-
NDCG — offline metric complementing click signals
-
Diversity Metrics — click diversity as diversity signal
-
Outcome-Based Relevance — relevance defined by clicks and purchases
-
CLEPR — an embedding model trained on session-stitched clicks
People
- Daniel Tunkelang — measuring searcher behavior; click signals in search evaluation
- James Rubinstein — “Measuring Search” — behavioral metrics
- Doug Turnbull — click residual metric