When There’s No Conversion Rate
Source: https://medium.com/@dtunkelang/when-theres-no-conversion-rate-67a372666fed Author: Daniel Tunkelang Series: Evaluating Search (Part 4 of 4)
Summary
What do you measure when your search product has no conversion signal? Long-document search, enterprise search, internal tools — these lack the clean purchase/signup events e-commerce relies on. This article proposes proxies.
The Problem
Some search domains lack reliable conversion signals:
- Long-document retrieval (academic, legal, enterprise)
- Internal search tools (no external revenue event)
- Exploratory browsing (goal is discovery, not a transaction)
In these contexts, “clicks still serve as a relevance signal, but they are only a weak signal.”
Alternative Metrics
- Dwell time — time on clicked result; longer = more engagement. Ambiguous for long documents (did they read it, or abandon it?)
- Session behavior — query repetition, pogo-sticking, progression to new queries; absence of reformulation signals satisfaction
- Client-side instrumentation — scrolling depth, in-page search usage; distinguishes browsing from actual reading
- Retention metrics — do users return? Better relevance increases search usage over time
The “Change the Problem” Insight
“Sometimes, when you can’t come up with a great solution, the better strategy is to change the problem.”
If click signal is too weak, improve snippets and autocomplete to strengthen the signal before trying to measure it.
User feedback buttons don’t work without giving users reciprocal value for their effort.
Series
- Evaluating Good Search - Measure It
- Measuring Searcher Behavior
- Evaluating Search - Using Human Judgments
- When There’s No Conversion Rate (this article)