Dense, hybrid and reranked: how much each search finds on the same knowledge base

Anyone building a RAG system picks between dense, hybrid and reranked search. Almost nobody measures the choice on their own data.

We measured it on ours before changing the knowledge-base search in CognitionDesk. The most useful result had more to do with the similarity threshold than with the reranker.

The benchmark

• Corpus: the docs at docs.cognitiondesk.com, split into 168 chunks. They're written in English.
• 48 questions in Spanish, phrased the way someone integrating the platform would ask them:
  • 20 with an exact term: a scope like orders:write, a signature header, an endpoint field.
  • 22 paraphrased: "My app keeps losing the session after a while: how long does the logged-in user's credential last?"
  • 6 with no answer in the corpus: "Does it support corporate single sign-on with SAML?"
• What counts as a hit: every answerable question carries its literal evidence. Recall@5 checks whether that text shows up in the top five results; MRR@5 also rewards it for ranking higher.
• Intervals: a 10,000-resample bootstrap over the questions. Differences between modes are paired, question by question.

The five modes:

1. Dense, threshold 0.65.
2. Dense, threshold 0.5.
3. Dense, no threshold.
4. Hybrid: dense plus keyword (BM25), fused with RRF.
5. Hybrid + rerank: gpt-4o-mini reorders 15 candidates and drops the ones that don't answer the question.

The first two are the thresholds our search tools use today.

How much each mode finds

With a 0.65 threshold, search finds the passage for 3 of 42 questions. With 0.5, for 15 of 42. With no threshold, plain dense search reaches 0.738.

Hybrid + rerank against dense at 0.5 gives +0.357 recall@5, interval [0.190 – 0.524]. The gain is real, and almost all of it comes from removing the threshold: dense search with no threshold is already at 0.738, without keywords or a reranker.

The threshold matters this much because of how people ask. The questions are in Spanish, the docs in English, and half the questions use different words than the docs. A fixed similarity threshold throws most of that away, even when the right passage is in the index.

Against dense with no threshold, hybrid stands out from the noise in only one place: MRR on exact-term questions, +0.15 [0.05 – 0.27]. When someone asks about orders:write, the keyword leg pushes the passage to the top. Dense search was already finding it.

So what is the reranker for?

Removing the threshold has a cost the first chart doesn't show.

Without a threshold, search never comes back empty, not even when there's nothing to find: all 6 unanswerable questions return passages. For an assistant, that's context unrelated to the question, and the model can use it to make up an answer. With a threshold you get the opposite: little noise, but silence on questions that did have an answer.

The reranker lets you drop the threshold without paying that price. It keeps hybrid's recall (0.714) and cuts the unanswerable questions that return something to 2 of 6. In exchange, 4 of the 42 answerable questions come back empty.

Latency and cost

mode	recall@5 [95% CI]	MRR@5 [95% CI]	p50 / p90 (ms)
dense, threshold 0.65	0.071 [0.000 – 0.167]	0.071 [0.000 – 0.167]	334 / 355
dense, threshold 0.5	0.357 [0.214 – 0.500]	0.306 [0.179 – 0.440]	244 / 342
dense, no threshold	0.738 [0.595 – 0.857]	0.516 [0.391 – 0.639]	242 / 257
hybrid	0.714 [0.571 – 0.833]	0.586 [0.452 – 0.717]	278 / 472
hybrid + rerank	0.714 [0.571 – 0.857]	0.645 [0.506 – 0.778]	1117 / 1327

Reranking 15 candidates with gpt-4o-mini costs USD 0.000414 per query (0.41 per thousand) and adds about 830 ms at the median. End to end, 1117 ms p50 against 244 for dense at 0.5.

What to do with your RAG

1. Before adding a reranker, measure your threshold. Write 40 or 50 questions with their literal evidence, in your users' language, with some paraphrased and some unanswerable. Run search with and without the threshold.
2. Measure both failure modes, not just recall: how many answerable questions come back empty, and how many unanswerable ones return something.
3. Compare question by question, with intervals. With 42 questions, the interval on a recall figure is around ±0.13. Dense with no threshold at 0.738 and hybrid at 0.714 doesn't tell you which one is better.

Limits of this measurement

• The benchmark is small: 168 chunks and a single corpus. On larger knowledge bases keyword search should matter more, and that isn't measured here.
• We wrote the questions ourselves, about our own docs.

In CognitionDesk

An assistant's knowledge base can search in dense, hybrid or hybrid+rerank mode. The mode is set per assistant and the default is still dense: nothing changes unless you pick another one.

If you want to run the same benchmark on your own knowledge base, write to us through the chat on this page.