Hybrid Search
Vector search fails on exact matches: product SKUs, error codes, legal citations, person names, and rare technical terms. BM25 fails on paraphrases. Hybrid search gives the best of both — the default architecture for production RAG in 2024+.
Hybrid search is like finding a restaurant using both 'similar cuisine' recommendations (semantic) and 'exactly named on the sign' lookup (keyword) — together you never miss the right place.
Visual Workflows
Start here — scroll inside each diagram frame to explore, then use + / − to zoom up to 200% if needed.
Overview
Scroll inside the frame to explore · use + / − to zoom up to 200%
Scroll inside the frame to explore · use + / − to zoom up to 200%
Two indexes, one list
Scroll inside the frame to explore · use + / − to zoom up to 200%
Run dense and sparse in parallel, then fuse ranks.
RRF idea
Scroll inside the frame to explore · use + / − to zoom up to 200%
Rank matters more than raw scores. 1 / (k + rank) lets both lists vote.
Key Takeaways
- 1.Hybrid search combines dense vector retrieval (semantic similarity) with sparse keyword retrieval (BM25) and fuses results — capturing both meaning-based matches and exact keyword/symbol matches that embeddings miss.
- 2.Pipeline: run vector search (top-k₁) and BM25 search (top-k₂) in parallel → fusion algorithm combines scores → optional re-ranker refines top results.
- 3.Fusion methods: Reciprocal Rank Fusion (RRF, most popular — rank-based, no score normalization needed), weighted linear combination (requires score normalization), and convex combination.
- 4.RRF score = Σ 1/(k + rank_i) where k=60 typically.
- 5.Production: tune k₁ and k₂ independently, A/B test hybrid vs pure vector on your eval set, and monitor which leg contributes to final results.
Real Example
Scenario
Query: 'error ECONNREFUSED in payment service'. BM25 matches the exact error code in logs. Vector search finds semantically related troubleshooting docs. RRF fusion surfaces both — neither alone would rank the combined result set optimally.
What you would do
In RAG Engineering, apply Hybrid Search to this scenario: Query: 'error ECONNREFUSED in payment service'. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.
Practice Task
Open the Code Walkthrough below and run it locally. Change one parameter related to Hybrid Search (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.
Code Walkthrough
Highlighted lines show where Hybrid Search happens in the code.
1def reciprocal_rank_fusion(results_lists: list[list[str]], k: int = 60) -> list[tuple[str, float]]: # define a reusable function2 scores: dict[str, float] = {}3 for results in results_lists:4 for rank, doc_id in enumerate(results):5 scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)6 return sorted(scores.items(), key=lambda x: x[1], reverse=True) # return the result7
8vector_ids = ["doc_a", "doc_c", "doc_b"] # from ANN search9bm25_ids = ["doc_b", "doc_d", "doc_a"] # from BM2510fused = reciprocal_rank_fusion([vector_ids, bm25_ids])11# doc_a and doc_b score highest (appear in both lists)Commands
Commands to Remember
Hybrid = vector search + BM25, then Reciprocal Rank Fusionscore = 1 / (60 + rank) # typical RRF kDo not add raw cosine to raw BM25 without normalizingUse hybrid when queries contain IDs, names, or error codes
Cheat Sheet
Quick recap
quick ref- •RRF k=60 default
- •Parallel vector + BM25
- •Vector=meaning, BM25=exact
- •Fuse top-20 → return top-10
- •Re-rank after fusion
- •A/B test before deploying
Common Mistakes
- ✕Averaging incomparable scores from vector and BM25 directly
- ✕Only using vector search for technical/support corpora
- ✕Not running searches in parallel — doubles latency unnecessarily
- ✕Skipping eval on keyword-heavy query subset
- ✕Fusing too many candidates without re-ranking — noise in context