Mateusz Sidor

Engineer at Allegro. Co-author of Allegro’s Relevance Assessment Tool (RAT), an LLM-as-a-judge framework for automating search relevance evaluation across Allegro’s multilingual e-commerce catalog.

Articles in This Vault

• Automating Search Relevance Assessment at Scale with LLM-as-a-Judge — Allegro’s RAT framework; multilingual annotation dataset, prompt design, local model migration

Concepts

• LLM as Judge
• Judgment Lists
• NDCG