Allegro

Allegro is an e-commerce platform operating a product catalog spanning 13 departments across four languages — Polish, Czech, Slovak, and Hungarian. Its engineering blog (Allegro Tech) has documented the team’s Relevance Assessment Tool (RAT), an LLM-as-a-judge framework for evaluating search relevance at catalog scale.

People

• Joanna Marhula — RAT framework co-author
• Mateusz Sidor — RAT framework co-author

Published Articles

• Automating Search Relevance Assessment at Scale with LLM-as-a-Judge — RAT framework; multilingual judgment dataset, prompt design findings, local model migration for cost