Implementing NDCG Scorer in Quepid
Quepid is a Test-Driven Relevancy Dashboard for relevancy practitioners — enables examining individual queries and system-level monitoring across test cases.
Quepid scorer components
- Descriptive name
- JavaScript code computing query scores
- Rating scale (default 1–10)
- Optional rating labels
NDCG@10 mathematics
CG: sum of ratings through rank position p
DCG: Σ rating(i) / log₂(i + 1) — logarithmic position discounting
nDCG: DCG / ideal_DCG → 0.0–1.0 scale, enabling cross-query comparison
Implementation steps
- Mock Quepid objects (
docs,bestDocs) in JSFiddle/CodePen for debugging - Create helper functions for rating retrieval and log calculations
- Implement
actualRatings()andidealRatings()functions - Compute DCG for both result sets
- Return ratio via
setScore()
Quepid integration
- Navigate to Custom Scorers → Add New
- Name (e.g., “NDCG@10”)
- Paste JavaScript implementation
- Select scorer in case settings
- Re-run evaluations
Note: “Number of Results to Show” setting must accommodate the desired rank position parameter; bestDocs capped at 10 results maximum.