Doug Turnbull
Co-founder of OpenSource Connections (OSC); currently independent (previously Shopify, then Reddit); co-author of Relevant Search (O’Reilly). Search relevance consultant and practitioner; author of the Softwaredoug blog.
Known For
- Relevant Search book (with John Berryman)
- Bayesian BM25 (BB25) — calibrated BM25 probabilities for principled hybrid fusion
- Practical Elasticsearch relevance tuning
- “What is Relevant?” — task-centered definition of relevance
Articles
-
Finding the Relevance Cutoff - When to Stop Showing Search Results
-
Getting Started on Search Relevance for the Understaffed Search Team
-
Principal Component Analysis - an embedding shrink-ray — PCA for embedding compression; measured recall vs. dimensions on MS MARCO
-
Just brute force your embeddings — measured NumPy scan throughput on 384-dim vectors; the case against adopting a vector database at ~1m documents
Affiliations
Teaching
- Courses — teaches Cheat at Search Essentials (free intro), co-teaches AI-Powered Search with Trey Grainger, and teaches Build your own vector database (https://maven.com/softwaredoug/vectordb)
Key Concepts
Affiliations
- OpenSource Connections (OSC) — Doug co-founded it but has since left (later Shopify, now independent); also former org of Max Irwin (now at Bonsai)
- Can BM25 be a Probability — BB25 framework; calibrating BM25 as probability for principled hybrid fusion
- Metadata - The 3rd Kind of Retrieval — attribute-based retrieval as a third paradigm beyond lexical + embeddings
- Classic ML to Cope with Dumb LLM Judges — combining many dumb LLM judges via decision tree to improve precision
- Don’t Classify, Hallucinate — hypothetical classifications: let a cheap LLM invent a category, resolve it into the real taxonomy by embedding similarity; see Query Classification
- Agentic search models — the case for Purpose-Built Agentic Search Models for “the last 20%”; see topic Frontier of Search 2026