RAG Glossary

An algorithmic approach for finding vectors close to a query vector without exhaustively comparing every vector. ANN algorithms (like HNSW, IVF, and Product Quantization) trade a small amount of accuracy for dramatically faster search speeds, making billion-scale vector search practical.

Best Matching 25 — a probabilistic ranking function used for keyword-based information retrieval. BM25 scores documents based on term frequency, inverse document frequency, and document length normalization. It excels at matching exact terms and identifiers, complementing dense vector search in hybrid RAG systems.

The process of splitting large documents into smaller, semantically meaningful pieces (chunks) before generating embeddings. Common strategies include fixed-size chunking, recursive character splitting, sentence-based splitting, and semantic chunking. Chunk size and overlap significantly affect retrieval quality.

The maximum number of tokens that a language model can process in a single request, including both the input prompt and the generated output. RAG systems must fit retrieved context within this window. Typical context windows range from 4K to 128K+ tokens depending on the model.

A metric for measuring the similarity between two vectors by computing the cosine of the angle between them. Values range from -1 (opposite) to 1 (identical direction). Widely used in vector databases for comparing embeddings because it is scale-invariant — it measures direction rather than magnitude.

A neural network model that takes a query-document pair as joint input and produces a relevance score. Cross-encoders are more accurate than bi-encoders for relevance scoring but are computationally expensive, making them suitable for reranking a small set of retrieved candidates rather than searching the entire corpus.

The first step in a RAG pipeline: ingesting raw documents from various sources (PDFs, web pages, databases, APIs, etc.) into a format suitable for processing. Document loaders handle format-specific parsing, text extraction, and metadata preservation.

Dense numerical vector representations of text (or other data) that capture semantic meaning. Similar texts produce similar embedding vectors, enabling semantic similarity search. Generated by embedding models like OpenAI's text-embedding-ada-002 or Sentence Transformers.

The process of further training a pre-trained language model on a specific dataset to adapt it for a particular task, domain, or output style. Unlike RAG, fine-tuning modifies the model's weights. The two approaches can be complementary: fine-tuning for style and task adaptation, RAG for dynamic knowledge access.

The practice of anchoring LLM-generated responses to verifiable source material. In RAG, grounding means the model's answer should be directly supported by the retrieved documents, with citations pointing back to the original sources.

When a language model generates information that sounds plausible but is factually incorrect, fabricated, or not supported by the provided context. RAG aims to reduce hallucination by grounding model responses in retrieved evidence.

A structured representation of knowledge as a network of entities (nodes) connected by relationships (edges). In GraphRAG, knowledge graphs enable relationship-aware retrieval, allowing the system to traverse connections between concepts and provide richer context than isolated document chunks.

A method for combining results from multiple retrieval systems (e.g., BM25 and vector search) by merging their ranked lists. RRF assigns scores based on the rank position of each document across different retrievers, producing a unified ranking that benefits from the strengths of each retrieval method.

A post-retrieval step that uses a more sophisticated model (such as a cross-encoder) to reorder retrieved documents by their actual relevance to the query. Reranking improves retrieval precision by re-evaluating the initial results from a faster but less accurate first-stage retriever.

An architecture pattern that enhances Large Language Model responses by first retrieving relevant information from external knowledge sources (such as documents or databases) and then providing that information as context for the model to generate more accurate, grounded answers.

The process of finding vectors (and their associated documents) that are most similar to a query vector. Common distance metrics include cosine similarity, Euclidean distance, and dot product. Approximate Nearest Neighbor (ANN) algorithms make this efficient at scale.

The basic units that language models process. Text is broken into tokens (roughly word pieces or subwords) before being fed to a model. Token count determines context window usage, API costs, and generation length. A rough rule of thumb is 1 token ≈ 4 characters in English.

A specialized database designed to store, index, and efficiently query high-dimensional vectors (embeddings). Examples include ChromaDB, Pinecone, Qdrant, Weaviate, and FAISS. Vector databases enable fast similarity search over large collections of embedded documents.