Basic / Naive RAG

What is Basic RAG?

Basic RAG (sometimes called Naive RAG) is the simplest and most straightforward implementation of Retrieval-Augmented Generation. It follows a linear pipeline: ingest documents, split them into chunks, generate embeddings, store them in a vector database, and at query time, retrieve the most relevant chunks to provide as context to an LLM. Example Scenario: A student uploads a Cloud Computing PDF and asks, "Explain virtualization." The system loads the PDF, finds the most relevant sections about virtualization, and generates a comprehensive answer based on the document's content. Basic RAG is the foundation upon which all other RAG approaches build. Understanding it thoroughly is essential before exploring more advanced patterns.

Step 1: Document Ingestion

Document ingestion is the process of loading raw documents and converting them into a format suitable for processing. Different document types require different loaders: PDF Documents: Libraries like PyPDF2, pdfplumber, or unstructured handle text extraction from PDFs. Some PDFs contain scanned images requiring OCR (Optical Character Recognition). Web Pages: HTML content needs to be parsed and cleaned, removing navigation elements, ads, and boilerplate to extract the main content. Structured Data: CSV, JSON, or database records may need transformation into text format suitable for embedding. Metadata Preservation: Good document loaders preserve metadata like page numbers, document titles, timestamps, and source URLs. This metadata is valuable for source attribution in the final answer.

document_loading.pypython

from langchain_community.document_loaders import PyPDFLoader
from langchain_community.document_loaders import WebBaseLoader

# Load a PDF document
pdf_loader = PyPDFLoader("cloud_computing.pdf")
pdf_docs = pdf_loader.load()
print(f"Loaded {len(pdf_docs)} pages from PDF")

# Each document has content and metadata
for doc in pdf_docs[:2]:
    print(f"Page {doc.metadata['page']}: {doc.page_content[:100]}...")

# Load a web page
web_loader = WebBaseLoader("https://example.com/article")
web_docs = web_loader.load()

Step 2: Text Chunking

Text chunking splits loaded documents into smaller pieces that are suitable for embedding and retrieval. The chunking strategy significantly impacts the quality of your RAG system. Key Parameters: • Chunk Size: The maximum size of each chunk (typically 200-1500 characters or tokens). Smaller chunks are more precise but lose context; larger chunks provide more context but may include irrelevant information. • Chunk Overlap: How much text is shared between consecutive chunks (typically 50-200 characters). Overlap helps preserve context at chunk boundaries. • Splitting Strategy: How the text is actually divided — by characters, sentences, paragraphs, or semantic boundaries. Common Strategies: • RecursiveCharacterTextSplitter: Tries to split by paragraphs, then sentences, then characters, preserving the most natural boundaries. This is the most commonly recommended approach. • SentenceTextSplitter: Splits at sentence boundaries, ensuring each chunk contains complete sentences. • SemanticChunker: Uses embeddings to identify topic boundaries and split at semantically meaningful points.

Choosing Chunk Size

Start with 500-1000 characters and 100-200 character overlap. Test with your actual queries and adjust based on retrieval quality. There is no universally optimal chunk size — it depends on your documents and questions.

text_chunking.pypython

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,       # Maximum chunk size in characters
    chunk_overlap=200,     # Overlap between consecutive chunks
    length_function=len,
    separators=["\n\n", "\n", ". ", " ", ""]  # Try these separators in order
)

chunks = splitter.split_documents(pdf_docs)
print(f"Created {len(chunks)} chunks from {len(pdf_docs)} pages")

# Inspect a chunk
print(f"Chunk size: {len(chunks[0].page_content)} chars")
print(f"Content preview: {chunks[0].page_content[:200]}...")
print(f"Metadata: {chunks[0].metadata}")

Step 3: Embedding Generation

Embeddings convert text chunks into dense numerical vectors that capture semantic meaning. Similar texts produce vectors that are close together in the embedding space. How Embeddings Work: An embedding model (a neural network) takes a text string as input and produces a fixed-length vector of floating-point numbers. For example, OpenAI's text-embedding-ada-002 produces 1536-dimensional vectors, while the smaller all-MiniLM-L6-v2 produces 384-dimensional vectors. Semantic Similarity: The key property of good embeddings is that semantically similar texts have similar vectors. "The cat sat on the mat" and "A feline rested on the rug" should produce vectors that are close together, even though they share few exact words. Choosing an Embedding Model: Consider: • Dimensionality (higher dimensions can capture more nuance but require more storage) • Speed (smaller models are faster) • Quality (benchmark performance on tasks similar to yours) • Cost (API-based vs. open-source local models)

embeddings.pypython

from langchain_openai import OpenAIEmbeddings
from sentence_transformers import SentenceTransformer

# Option 1: OpenAI embeddings (API-based)
openai_embeddings = OpenAIEmbeddings(model="text-embedding-ada-002")
vector = openai_embeddings.embed_query("What is virtualization?")
print(f"OpenAI embedding dimension: {len(vector)}")  # 1536

# Option 2: Open-source embeddings (local)
model = SentenceTransformer('all-MiniLM-L6-v2')
vectors = model.encode(["What is virtualization?", "Cloud computing basics"])
print(f"Local embedding dimension: {vectors.shape[1]}")  # 384

# Compute similarity
from numpy import dot
from numpy.linalg import norm
similarity = dot(vectors[0], vectors[1]) / (norm(vectors[0]) * norm(vectors[1]))
print(f"Cosine similarity: {similarity:.4f}")

Step 4: Vector Storage & Retrieval

The generated embeddings are stored in a vector database alongside the original text and metadata. When a query arrives, the system embeds the query and performs similarity search to find the most relevant chunks. Storage: Vector databases use specialized indexing structures (like HNSW — Hierarchical Navigable Small World graphs) to enable fast approximate nearest neighbor search over millions or billions of vectors. Retrieval: At query time: 1. The user's question is embedded using the same embedding model 2. The vector database finds the top-k most similar vectors using cosine similarity or other distance metrics 3. The corresponding text chunks and metadata are returned Filtering: Most vector databases support metadata filtering, allowing you to narrow searches by source document, date range, category, or other attributes.

vector_store.pypython

from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()

# Create vector store from chunks
vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="./chroma_db",
    collection_name="cloud_computing"
)

# Retrieve relevant chunks
retriever = vectorstore.as_retriever(
    search_type="similarity",
    search_kwargs={"k": 4}  # Return top 4 most relevant chunks
)

# Search
results = retriever.invoke("Explain virtualization")
for i, doc in enumerate(results):
    print(f"\nResult {i+1} (Page {doc.metadata.get('page', '?')}):")
    print(doc.page_content[:200])

Step 5: Answer Generation

The final step combines the user's question with retrieved context and sends it to an LLM for answer generation. The prompt template is crucial — it instructs the model to use only the provided context and to acknowledge when the answer isn't available in the retrieved documents. Prompt Template Best Practices: • Clearly instruct the model to base its answer on the provided context • Tell the model to say "I don't have enough information" rather than hallucinate • Include the source metadata for citation purposes • Keep the prompt focused — don't overload with instructions Complete Basic RAG Chain: The full pipeline connects all components into a single retrieval chain that takes a question and returns an answer with sources.

About chain_type

The "stuff" chain type puts all retrieved documents into a single prompt. For large numbers of retrieved documents, consider "map_reduce" (summarize each document then combine) or "refine" (iteratively improve the answer with each document).

basic_rag_chain.pypython

from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

# Define the prompt template
prompt_template = """Use the following context to answer the question. 
If the answer is not in the context, say "I don't have enough information to answer this question."

Context:
{context}

Question: {question}

Answer:"""

PROMPT = PromptTemplate(
    template=prompt_template,
    input_variables=["context", "question"]
)

# Create the QA chain
llm = ChatOpenAI(model="gpt-4", temperature=0)
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",  # Stuff all retrieved docs into the prompt
    retriever=retriever,
    return_source_documents=True,
    chain_type_kwargs={"prompt": PROMPT}
)

# Ask a question
result = qa_chain.invoke({"query": "Explain virtualization in cloud computing"})
print("Answer:", result["result"])
print("\nSources:")
for doc in result["source_documents"]:
    print(f"  - Page {doc.metadata.get('page', '?')}")

Limitations of Basic RAG

While Basic RAG is a solid starting point, it has several limitations that more advanced approaches address: No Conversation Memory: Basic RAG treats each question independently. It cannot handle follow-up questions like "What about its advantages?" because it doesn't know what "its" refers to. Conversational RAG addresses this. Single Retrieval Method: Basic RAG uses only vector similarity search. It may miss documents that match by exact keywords (like error codes) but aren't semantically similar. Hybrid RAG combines keyword and vector search. No Quality Control: Basic RAG doesn't evaluate whether the retrieved documents are actually relevant. It simply returns the most similar vectors, which may not contain the answer. Corrective RAG and Self-RAG add evaluation steps. No Query Refinement: If the initial query doesn't retrieve good results, Basic RAG has no mechanism to rewrite the query or try alternative search strategies. Agentic RAG adds this capability. Text Only: Basic RAG handles text documents. It cannot process images, charts, or tables within documents. Multimodal RAG extends the approach to multiple modalities. Despite these limitations, Basic RAG is remarkably effective for many use cases and should be your starting point before adding complexity.

Learning Objectives

What is Basic RAG?

Step 1: Document Ingestion

Step 2: Text Chunking

Step 3: Embedding Generation

Step 4: Vector Storage & Retrieval

Step 5: Answer Generation

Limitations of Basic RAG

Key Takeaways