How we Compute NDCG in Daraz (E-Commerce)
NDCG evaluates ranking quality through three principles: relevance hierarchy, position weighting, and query independence. Selected as “the most simple and widely used measure in ranking evaluation.”
Relevance weights from user funnel
| Action | Score |
|---|---|
| Purchase | 5 |
| Add to Cart | 2 |
| Click | 1 |
Implementation assumptions
- Top-10 only — lower positions yield negligible impact
- Clicked-only instances — focus on searches with at least one product interaction
- A/B-test ready — supports comparison across multiple ranking algorithms
Calculation
- DCG: weighted relevance sum with position-based discount factors
- IDCG: same calculation on perfectly-ranked items based on actual user behavior
- NDCG = DCG / IDCG
Implementation: Python UDFs for individual search instance calculation + SQL aggregation across datasets.