Hybrid Search: Combining Semantic and Keyword Retrieval for Better RAG Results

Anyone who has spent time building RAG systems knows this pretty quickly: your model’s answers are only as good as what you feed it. You can have the smartest language model in the world, but if it’s working with the wrong context, you’re going to get answers that sound confident but miss the mark. That’s the whole game with retrieval. It’s the step where the system goes and finds the right documents or text chunks before the model even starts writing a response.

Most people building these systems start with one of two approaches, either keyword search or semantic (vector) search. Both have their moments where they shine, and both have moments where they just don’t cut it. That gap is basically why hybrid search RAG has become such a hot topic lately, especially for companies like Impressico Business Solutions that work on building serious enterprise search and AI tools. Let’s get into what hybrid search actually means, how it functions, and when combining the two methods can outperform picking just one.

Keyword search, or lexical search if you want the technical term, is the old school way of finding stuff. It matches the actual words in your search query against the words sitting in a document. This has been around for a long time, way before anyone was talking about AI-powered search.

The algorithm doing the heavy lifting here is usually BM25, or sometimes its older relative TF-IDF. The basic idea is this: BM25 checks how often a word shows up in a document, and also how rare that word is across your whole collection of documents. So if a word is uncommon overall but shows up a bunch in one particular document, that document jumps up in the rankings.

How it actually works: First, the text gets broken down into individual words or tokens. Then each word gets checked against an index that was built from all your documents ahead of time. Whichever documents contain the exact query words end up ranked higher, based on how often those words appear and how rare they are.

Semantic search takes a completely different approach, and you’ll often hear it called vector search instead. Rather than looking for matching words, it tries to figure out what the query actually means. It does this by turning text into something called an embedding, which is basically a long string of numbers that represents the meaning of a sentence or paragraph.

Here’s the process: The query and the documents both get run through an embedding model. That model spits out vectors, which you can think of as points scattered across a huge multi-dimensional space. From there, the system looks for documents whose vectors sit closest to the query’s vector, usually measured using something called cosine similarity. Documents that mean similar things end up clustered close together in this space, even if they don’t share many actual words.

Now here’s where it gets tricky. Neither of these approaches works perfectly on its own, and honestly, both have some pretty obvious blind spots.

Figure 1 — Where each method succeeds, where it fails, and why hybrid exists.

Pure semantic search can miss exact matches. Got a specific product code, a part number, an acronym, or some numeric value buried in your documents? Vector search might just skip right past it, because embeddings are designed to capture meaning, not exact precision. Sometimes two totally different numbers end up looking “similar” in vector space just because they show up in similarly structured sentences.

Then flip it around. Pure keyword search, whether that’s BM25 or old-school TF-IDF, does not model meaning or paraphrasing directly. If someone phrases their question differently than how it’s written in your source documents, keyword search might come back with nothing useful at all, even though the answer is sitting right there in your data.

This is basically the whole reason people keep bringing up hybrid search vs vector search in AI and RAG circles.

So what is hybrid search in RAG, really? Put simply, it’s a retrieval approach that uses both lexical (keyword) and semantic (vector) signals at the same time, then combines the results from both into one better-ranked list.

Instead of forcing you to pick a side, hybrid search grabs the strengths from each method. The keyword piece catches your exact terms, codes, and names. The semantic piece catches meaning and context. Put together, the aim is retrieval that is both precise and flexible, which is what many real-world use cases need.

RAG systems get used everywhere these days, legal, healthcare, e-commerce, enterprise support, you name it. And if you look at the actual documents these industries deal with, they’re usually a mix of technical jargon, product codes, dates, and plain conversational language, often all crammed into the same knowledge base.

Think about a customer support article that explains a return policy in plain English but also lists an exact SKU number somewhere in there. Or a legal contract that mixes formal legal language with specific section numbers. A retrieval system that can only understand meaning, or only match exact words, is likely to struggle with part of that document.

This is exactly the kind of mixed content where hybrid search RAG can help. Better retrieval gives the language model more relevant context to work with, which creates the conditions for more accurate, grounded answers — though answer quality and hallucination rates still need to be evaluated separately rather than assumed.

Once you’ve got two separate sets of results, one from keyword search and one from semantic search, you need some way to merge them into a single ranked list. There are two methods that come up most often.

A. Weighted Score Fusion

With this approach, each system assigns a score to every document it returns. Keyword search hands out a BM25 score, semantic search hands out a similarity score. Then you combine both scores using a weighted formula, something along the lines of:

The tricky part is that BM25 scores and cosine similarity scores don’t live on the same scale at all. So before you can combine them fairly, you usually have to normalize both sets of scores first, otherwise one method ends up dominating the results just because its numbers happen to run higher. Teams usually spend a fair amount of time testing different weight combinations to see what actually works for their specific data.

B. Reciprocal Rank Fusion (RRF)

This brings up a question that comes up constantly: what is reciprocal rank fusion (RRF)? Introduced by Cormack, Clarke and Buettcher at SIGIR in 2009, RRF takes a different tack. Instead of messing with raw scores from two different systems, it only looks at where each document landed in each list, meaning its rank or position.

The formula looks like this:

Here, rank_r(d) is document d’s position in ranked list r — first place, second place, and so on — and the Σ means those reciprocal values are summed across every list the document appears in. In a two-retriever hybrid setup, that is one term from the keyword list plus one from the semantic list. “k” is a smoothing constant that softens the impact of very high rankings. A value of 60 is commonly used, and is the default in the original paper and many implementations, but it is a tunable parameter rather than a fixed value.

The practical advantage of RRF is that it combines rank positions rather than raw scores, so it avoids the need to directly normalize or compare BM25 and similarity scores that sit on different scales. That’s probably why so many RAG systems these days lean toward RRF over weighted fusion, particularly teams who want solid results without spending weeks fine-tuning weights. The trade-off is worth knowing: in OpenSearch’s own benchmark across six BEIR datasets, RRF scored 3.86 percent below score-based fusion on nDCG@10. That figure is specific to their benchmark and configuration rather than a general result. The broader pattern is that RRF offers convenience and robustness, while a well-tuned weighted setup may reach a higher ceiling if you have the evaluation data to get there — and the difference should be measured on your own dataset.

Figure 2 — The two fusion methods side by side, and what each one costs.

Here’s roughly what a hybrid search setup looks like inside a typical RAG pipeline:

Figure 3 — The seven steps described below, as a pipeline.

Document ingestion: Your documents get split into chunks and stored in two separate places at once, a traditional search index like Elasticsearch or OpenSearch for keyword search, and a vector database like Pinecone, Weaviate, or Qdrant for the embeddings.

Query time: When someone asks a question, that same query gets fired off to both systems simultaneously.

Keyword retrieval: The keyword index sends back a list of top matches based on BM25 scoring.

Semantic retrieval: The vector database sends back its own list based on embedding similarity.

Fusion step: Both lists get merged together, either through weighted score fusion or RRF.

Re-ranking (optional): Some setups throw in an extra re-ranking model at this point to sharpen the final ordering before it goes anywhere near the language model.

Generation: The top chunks get handed off to the LLM as context, and that’s what generates the actual answer the user sees.

This setup is designed to draw on the strengths of both retrieval methods rather than forcing a choice between them.

If you’re asking whether hybrid search is better than vector search alone, the honest answer is “it depends,” but it can perform better when both lexical and semantic signals matter to your queries — and the only reliable way to know is to validate it on your own dataset.

Hybrid search is most likely to help when your documents contain a mix of precise terminology and natural, conversational language, which is common in business knowledge bases. Picture internal documentation that combines product SKUs with plain-English explanations, or medical records that mix exact drug codes with descriptive notes about how a patient is feeling. In situations like these, vector search alone can miss exact codes, and keyword search alone can miss the natural language questions people actually type. Hybrid search is designed to draw on both signals.

Now, if your data is extremely uniform, say it’s all technical codes or all free-flowing prose, a single method might do the job just fine. But most business content is messy and mixed together, and that messiness is exactly the problem hybrid search was designed to solve.

Hybrid search brings a lot to the table, but it’s not without its downsides. Worth keeping these in mind:

All of this sounds good on paper, but how do you actually prove it works? That comes down to measuring both the quality of what gets retrieved and the quality of the final answer the whole RAG system produces.

Retrieval metrics tell you how good your retrieved documents are before they even reach the language model:

RAG-level metrics go further and look at the quality of the actual generated answer:

Published benchmarks show that hybrid setups can outperform semantic-only or keyword-only search on recall and nDCG, particularly on datasets with mixed content types — but the size of the gain depends heavily on configuration. On the WANDS e-commerce benchmark, the roughly 7.4 percent higher nDCG (reaching 0.7497) came from the final tuned hybrid configuration, which added product-name boosting on top of hybrid retrieval, rather than from basic hybrid search or RRF alone. On financial documents mixing text and tables, hybrid RRF on its own reached Recall@5 of 0.695 against 0.587 for dense-only retrieval; adding neural reranking on top lifted that to 0.816. Retrieval gains like these do not automatically translate into better faithfulness or groundedness, so those need to be evaluated separately on the full RAG output — and any of these results should be validated on your own dataset before you rely on them.

Retrieval is really the backbone of any solid RAG system, and the retrieval strategy you pick has a massive impact on the quality of answers your AI ends up giving. Pure keyword search and pure semantic search both bring real strengths to the table, but they also both come with real blind spots. Hybrid search, by pairing BM25 with vector search and using fusion techniques like reciprocal rank fusion, gives you a retrieval setup designed to handle both exact terminology and natural language questions — though how well it does so on your own data is something to measure rather than assume.

For businesses building serious AI-powered search or RAG applications, hybrid search is worth evaluating whenever both exact terminology and natural language matter to your users — with the benefit confirmed on your own data rather than assumed. At Impressico Business Solutions, we help companies design and build retrieval pipelines made for messy, real-world data, not just tidy test cases, so the AI systems you rely on give answers your team can actually trust.