IntermediateEstimated time: 40 mins

Hybrid Search (BM25 + Dense Vectors)

Combine lexical keyword search (BM25) with semantic dense vector search using Reciprocal Rank Fusion (RRF).

Tech Stack:PythonRank-BM25ChromaDB / FAISSLangChain EnsembleRetriever

System Architecture

Query ➔ [BM25 Keyword Retriever + Dense Vector Retriever] ➔ Reciprocal Rank Fusion (RRF) ➔ Top Re-ranked Chunks ➔ LLM

Prerequisites

  • Understanding of TF-IDF / BM25 term weighting
  • Understanding of Dense Embedding Cosine Distance

Installation Commands

terminalbash
pip install rank-bm25 chromadb langchain langchain-community langchain-openai

Folder Structure

Directory Layouttext
hybrid-rag/
├── hybrid_pipeline.py
└── sample_docs.json

Step-by-Step Implementation

1

Initialize BM25 and Dense Retrievers

Create both BM25Retriever (for precise keyword matches, acronyms, and error codes) and VectorStoreRetriever (for conceptual similarity).

hybrid_pipeline.pypython
from langchain_community.retrievers import BM25Retriever
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain.retrievers import EnsembleRetriever
from langchain_core.documents import Document

docs = [
    Document(page_content="Error code 0x80070005 indicates Access Denied in Windows permissions."),
    Document(page_content="Permission issues often occur when services lack administrative privileges."),
    Document(page_content="Database connection timeout error 504 gateway response from Nginx."),
]

# 1. BM25 Lexical Retriever
bm25_retriever = BM25Retriever.from_documents(docs)
bm25_retriever.k = 2

# 2. Dense Semantic Vector Retriever
embeddings = OpenAIEmbeddings()
chroma_db = Chroma.from_documents(docs, embeddings)
dense_retriever = chroma_db.as_retriever(search_kwargs={"k": 2})

# 3. Ensemble Retriever with Reciprocal Rank Fusion (50% weight each)
ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, dense_retriever],
    weights=[0.5, 0.5]
)

# Test exact keyword query
results = ensemble_retriever.invoke("0x80070005")
for r in results:
    print(r.page_content)

Example Execution (Input & Output)

Input Query:
Query: "0x80070005 access error"
Generated Grounded Output:
Rank 1 Document: "Error code 0x80070005 indicates Access Denied in Windows permissions." (Score boosted by exact BM25 match)

Why this works: Dense embeddings often struggle with arbitrary alphanumeric error codes, but BM25 pins them directly.

Troubleshooting Common Gotchas

BM25 returns irrelevant results due to common stop words

Configure custom tokenizers or stopword filters in Rank-BM25.

Known Limitations

  • •Requires maintaining both an inverted keyword index and a vector index in sync.

Security Considerations

  • •Ensure metadata filtering applies equally to both the BM25 index and the vector store.
Back to all projects