Search Relevance for Understaffed Teams

Source: https://softwaredoug.com/blog/2023/05/29/guide-for-search-teams.html Author: Doug Turnbull

Summary

Practical guide for search teams with limited resources. Core thesis: resource constraints force strategic clarity — they’re an opportunity to focus on high-impact work rather than a blocker.

Key Philosophy

“It forces you to focus on value NOW, not some distant future after we’ve hired a ton of new people.”

Limited teams should resist building elaborate infrastructure and instead identify the smallest interventions with the highest ROI.

Recommended Approach

• Community-driven — Turnbull compiled approaches via public Google Doc from practitioners
• Wiki-style living document of proven methods
• Focus on quick wins achievable without headcount growth

Practical Relevance Work Principles

1. Quick iterations over long projects — ship improvements weekly, not quarterly
2. Evaluation first — build Judgment Lists before building models
3. Automate what repeats — Search Evaluation harness to catch regressions
4. Rules before ML — deterministic fixes often outperform learned models on sparse data

Related Concepts

• Search Evaluation
• Judgment Lists
• Learning to Rank
• A-B Testing for Search

People

• Doug Turnbull

Related Articles

• What Is a Judgment List
• Measuring Search - Metrics Matter
• Query Triage - The Secret Weapon for Search Relevance
• Getting Started on Search Relevance for the Understaffed Search Team — full strategy guide (Google Doc version, more comprehensive)