James Rubinstein

Search product manager and practitioner; advocate for rigorous evaluation methodology and data-driven search product development.

Known For

• Relevance evaluation programs — 6-step methodology for setting up systematic relevance judgment
• Search PM philosophy — “everything in search is an experiment”
• Statistical and human-centered frameworks for search improvement
• Baseball metaphor for search quality (batting average of a search system)

Articles

• Evaluating Search - Using Human Judgments
• Setting Up a Relevance Evaluation Program
• Statistical and Human-Centered Approaches to Search Improvement
• Search Product Management - The Most Misunderstood Role

Key Concepts

• Search Evaluation
• Judgment Lists
• A-B Testing for Search
• Search Architecture
• Query Understanding