Awesome Search KG

Oops, I Did It Again

2 min read

Oops, I Did It Again

The sequel to Why Setting Up Quepid for Vector Search Evaluation Went Wrong. Having hit query-length and JSON-validity walls, Andrew Kornilov finds a workaround that supports much higher-dimensional vectors: stop typing queries into Quepid one by one, and instead generate a complete case file with embeddings baked in.

The Approach

Rather than adding queries through the UI, generate a Quepid-compatible case JSON that contains, per query:

  • the query text, and
  • its associated embedding (OpenAI text-embedding-3-large, 94 dims — see Matryoshka Embeddings)

Eight sample product queries are used (e.g. “Bushcraft Survival Manual”, “Scrub Top”).

Steps

  1. Generate embeddings via the OpenAI API
  2. Structure queries with embedded vectors in Quepid-compatible JSON
  3. Upload the case file to Quepid
  4. Configure the Elasticsearch backend to use the vectors in search requests
  5. Begin the relevance judgment workflow

By carrying the vector inside the case’s query options, the per-query input limit that broke the first attempt no longer applies — enabling higher dimensionality.

Limitations & What’s Next

  • Storing embeddings in query options currently requires generating a whole case file — there’s no per-query API to add one at a time. The author is building an unofficial Quepid API to close that gap (later realized as quepid-api-unofficial, published as the frutik777/quepid-api-unofficial Docker Hub image).
  • The same technique extends beyond pure semantic search to Hybrid Search and vector-based re-ranking.

People

Tools

Graph View

Backlinks