History — 2026 week 41 (Oct 5 – Oct 11, 2026)
Newest first.
2026-10-11 — Cheap signals for e-commerce ranking (1 new, 0 updated)
Many of the fixes that last in e-commerce search are built from data the shop already has: conversion logs, click positions, stock counts and session rephrasings. Cheap Signals, Shipped is a guide to Marcos Rebelo’s ongoing series making that case. The seven posts it picks out cover query-to-category priors that filter retrieval and guard vector search (Query Classification), a click score corrected for position bias, a stock boost measured in days of stock, sold-out items that fade instead of vanishing, synonyms mined from session reformulations, and a weekly review of the queries losing the most clicks. The original posts are member-only on Medium, so only the ideas are summarised here.
Articles — Cheap Signals, Shipped (guide to Marcos Rebelo’s search series)
2026-10-11 — Voyage rerank-3 and the long-document push (4 new, 10 updated)
Reranker traffic is changing: coding agents now send many of the queries, and the candidate documents are getting longer. Voyage AI’s Voyage rerank-3 and rerank-3-lite target both. They keep rerank-2.5’s price, 32K context and instruction-following, and keep its score distributions, so tuned thresholds survive the upgrade (Calibrated Relevance Probability). On Voyage’s own 95-dataset suite, rerank-3 leads Cohere Rerank v4.0 Pro by 2.72% NDCG@10, and by 13.84% on LongEmbed long documents. A partial, untuned comparison has it ahead of Jev in code, law and finance. Every figure comes from the vendor’s launch post.
Corrections
- Hev meets Jev, Hev, Reception of Jev and Voyage AI called Voyage rerank-3 the best reranker in Hev’s comparison → the published post’s leader is Mixedbread’s mxbai-rerank-v3.1-listwise at 0.511; untuned Jev is 0.003 behind rerank-3 and 0.010 behind the leader.
Articles — rerank-3 and rerank-3-lite - the next generation of Voyage rerankers Tools — Voyage rerank-3 Datasets — LongEmbed · MAIR (instruction-following retrieval benchmark) Updated — Voyage AI · Cohere · MongoDB · Jev (second rerank-3 comparison) · Calibrated Relevance Probability (score distributions held across versions) · Reranking · Reasoning Reranking · Hev meets Jev (missing leader row added) · Hev · Reception of Jev
2026-10-11 — Calibrating an AI judge before trusting it (2 new, 8 updated)
How good an AI relevance judge is depends on the model, the prompt, the collection and the reference it is compared with, all together. Judge Calibration covers measuring a judge against a reference that has already rated a frozen random sample of pairs. Agreement is reported both per pair, with Krippendorff’s α, and per ranking, with Kendall’s τ over per-query nDCG@10, because the two levels decouple. Prompt variants are compared on the same pairs. With escalating judges, calibration runs as a chain from people to the expensive judge to the cheap one (Staged Judging). From Andrew Kornilov’s essay on a Quepid calibration tab that was still a proposal when the essay was written.
Articles — Calibrate Before You Trust Concepts — Judge Calibration Updated — Andrew Kornilov · Levels of Judge Agreement · Staged Judging (calibrate each stage) · Quepid · LLM as Judge · Inter-Annotator Agreement · Prompt Sensitivity (same-pair comparisons) · Do LLM Judges Actually Agree With Us
2026-10-11 — Jev beyond the single relevance question (7 new, 16 updated)
Search work with Jev has moved past asking one “is this relevant?” question. Elastic asks for ESCI-relationship probabilities plus constraint signals and lets a short policy score them. On 250 ESCI queries, where random order already scores 0.86, nDCG@10 rose from 0.9351 to 0.9565. Doug Turnbull has an LLM write a yes/no rubric for each query and sums the answers (Bag of Decisions). Qdrant applies the same model to MMR diversity, a tiered category filter/boost, and topic-boundary chunking evaluated on QASPER. Qdrant also shows apple filtered to food at 0.91 confidence. Each is a single experiment run by its authors on their own benchmarks.
Articles — Using Jev as a Search Reranker - Benchmarks and How to Implement · Bag of Decisions Reranker · Jevjitsu - How We Tried Generalized Classifiers on Everything Concepts — Bag of Decisions (per-query yes/no rubric as rerank score) People — Dustin Coates · Chadha Sridi Datasets — QASPER Updated — Jev · Reception of Jev · Reasoning Reranking · Reranking · ESCI Label Scheme (expected-utility scoring) · MMR (reranker score as relevance term) · Text Chunking (boundary classification) · Search Result Diversity · Query Classification (confidence-tiered filter/boost) · WANDS Dataset · Amazon ESCI Dataset · Doug Turnbull · Andrei Cristea · Qdrant · Elastic · TypeSafe
2026-10-05 — Semantic and outcome-based relevance compared (1 new, 6 updated)
Whether a result matches the query and whether people buy it are different questions, and the answers can move apart: at Etsy, engagement sometimes falls as semantic relevance rises. Outcome-Based vs Semantic Relevance sets the two side by side. It covers where each label comes from (judges and ESCI grades against clicks and conversions), why they diverge (popularity, Position Bias, items nobody saw), and four ways systems combine them. Semantic match can gate candidates that commerce signals then order, as at Criteo. Learning to Rank can blend text-match features under behavioural labels, where the choice of label decides which relevance wins. Clicks can train a semantic model, and behaviour can decide which pairs get judged at all.
Topics — Outcome-Based vs Semantic Relevance Updated — Outcome-Based Relevance · Semantic Relevance · Learning to Rank · Ranking Signal Selection · E-commerce Search · Two-Sided Marketplace Ranking
2026-10-05 — Outcome-based relevance and Criteo’s CLEPR (6 new, 14 updated)
A product can match a query on brand, category and attributes and still not be what shoppers buy. Outcome-Based Relevance covers relevance judged by clicks and purchases, set against Semantic Relevance, and CLEPR covers Criteo’s two-tower query–product model, trained on clicks rebuilt from session logs. CLEPR gates sponsored-product ads on semantic match. In Criteo’s offline test over 400 retrieved candidates, re-ranking by recent sales share raised the purchased product’s normalized rank score about 2.5×. From posts by Maxime Vono and Paul Coursaux; both are member-only, and every figure is Criteo’s own.
Articles — Leveraging Commerce Data for Outcome-Based Relevancy in Agentic Recommendation Systems · Introducing CLEPR, our model for semantic understanding Concepts — Outcome-Based Relevance People — Maxime Vono · Paul Coursaux Tools — CLEPR (Criteo’s query–product embedding model) Updated — Criteo · Semantic Relevance (Criteo’s accuracy vs outcome framing) · Click Signals (clicks stitched from sessions) · Implicit Judgments · Bi-Encoder · Contrastive Learning · Position Bias · Embedding Fine-tuning · Reranking · MRR · E-commerce Search · Conversational and Agentic Search · Model Selection and Fine-Tuning Evaluation · Multilingual Search