Awesome Search KG

How to Securely Hook Up Quepid to Vespa

4 min read

How to Securely Hook Up Quepid to Vespa

Charlie Hull (The Search Juggler) documents a proof-of-concept for connecting Quepid — the open-source relevance workbench from OpenSource Connections — to a Vespa application. Vespa is “hugely flexible and powerful” but approaches search differently from the Lucene-based engines Hull usually works with, and crucially it lacks an easy, interactive way to tune queries offline. Quepid fills that gap, but getting the two to talk securely is non-obvious.

Companion resource

Hull also co-presented a Maven “Lightning Lesson” with Trey Grainger — Offline search relevance testing with Vespa & Quepid — covering the same territory in video form.

Why This Matters

Vespa ships some offline evaluation tools in Python, but nothing that gives a relevance engineer a fast, interactive loop over test queries and human judgements. Quepid provides exactly that for many engines as a free, open-source-backed service — the challenge is that it isn’t obvious how to extract the data Quepid needs from Vespa’s API, or how to authenticate against a cloud deployment. This is a concrete instance of driving Quepid against a custom search API rather than its native Elasticsearch/Solr integrations.

The Setup

1. Vespa Cloud (not localhost)

Quepid cannot connect to a localhost HTTP server, so Hull deployed to Vespa Cloud using its free $300 developer credits rather than self-hosting. He used the Music example application (5 album documents), deployed via Docker and the Vespa CLI on Windows PowerShell.

2. Token authentication (not certificates)

Quepid cannot use Vespa’s default self-signed client certificates, so Hull switched to read-only token authentication:

  • Generate an auth token in the Vespa Cloud console.
  • Edit services.xml to add a <clients> block granting read permission to a token client, while keeping certificate-based access for CLI operations via a second client definition.
<clients>
  <client id="query-token-client" permissions="read">
    <token id="vespaquepid1"/>
  </client>
</clients>

3. Redeploy on every change

A key gotcha: every configuration change requires re-deploying the Vespa application.

vespa auth cert -f
vespa deploy --wait 600
vespa feed ../dataset/documents.jsonl

4. Verify with cURL

Before touching Quepid, Hull confirmed token auth works by querying Vespa with YQL and a Bearer token authorization header, checking the JSON response.

5. Quepid Custom Search API endpoint

In Quepid he created a Custom Search API endpoint holding:

  • the Vespa Cloud application URL,
  • an Authorization header carrying the token (in JSON format),
  • JavaScript parsing functions mapping Vespa’s response into Quepid’s document model:
    • numberOfResultsMapper — extracts the total result count,
    • docsMapper — transforms Vespa hits into docs with id, title, artist, year, and relevance score.

6. Query with YQL

Quepid sends queries using YQL syntax, with a placeholder token replaced by each test query.

Future Directions

Hull suggests exploring more complex YQL queries, larger datasets with multiple result documents, and — notably — exporting Quepid ratings as training data for Vespa’s re-ranking, closing the loop from offline judgement to learned ranking. He reiterates that Vespa lacks built-in offline testing facilities, making Quepid a valuable complementary tool.

  • Search Evaluation — the offline judgement → metric loop this enables for Vespa
  • Judgment Lists — what you build in Quepid once it’s wired up
  • NDCG — the metric you’d score Vespa results against

People

Tools

Graph View

Backlinks