🔍 Retrieval Augmented Generation (RAG)

🌟 Start With a Real-World Scenario

You ask ChatGPT about your company:

You: "What's our company's vacation policy?"

ChatGPT: "I don't have access to your company's specific policies.
I can only provide general information about vacation policies..."

❌ Useless! ChatGPT doesn't know about YOUR company!

You try to help:

You: "Our vacation policy is in the employee handbook, page 23,
section 4.2. It says employees get 15 days PTO, 10 sick days,
and 8 holidays. New employees get 10 days PTO in their first year.
PTO accrues monthly. Unused PTO rolls over up to 5 days..."

ChatGPT: "Based on your company policy, employees receive 15 days
of PTO annually, with new employees receiving 10 days in their
first year..."

✅ Works! But you had to copy-paste the entire policy! 😤

Problems:

• You need to find the right document
• You need to copy-paste it
• Context window limits (can't paste 100 pages!)
• Every question needs manual work

Enter RAG (Retrieval Augmented Generation):

You: "What's our vacation policy?"

RAG System:
1. 🔍 Searches your company docs
2. 📄 Finds: "Employee Handbook, page 23, section 4.2"
3. 🤖 Sends to ChatGPT with context
4. 💬 ChatGPT answers based on YOUR docs

ChatGPT: "According to your Employee Handbook (page 23),
employees receive 15 days of PTO annually. New employees
receive 10 days in their first year. PTO accrues monthly
and up to 5 days can roll over to the next year."

✅ Perfect! Automatic, accurate, sourced!

The RAG Revolution:

Before RAG (2020):

You: "What did the CEO say in last week's all-hands meeting?"
ChatGPT: "I don't have access to recent information..."

❌ Training cutoff problem!

With RAG (2023):

You: "What did the CEO say in last week's all-hands meeting?"

RAG System:
1. Searches meeting transcripts
2. Finds last week's all-hands
3. Retrieves relevant sections

ChatGPT: "In last week's all-hands (March 1, 2024), the CEO
announced three key initiatives:
1. New product launch in Q2
2. Expansion to European markets
3. 20% headcount growth this year

Source: All-Hands Meeting Transcript, March 1, 2024"

✅ Up-to-date, accurate, with sources!

Real-World Impact:

1. Customer Support (Intercom, Zendesk)

Without RAG:

• Agent searches knowledge base manually
• Copies relevant articles
• Pastes into response
• Time: 5-10 minutes per ticket
• Accuracy: 70% (human error)

With RAG:

• AI searches automatically
• Finds relevant articles
• Generates response with sources
• Time: 10 seconds per ticket
• Accuracy: 95%

Result: 30x faster, 25% more accurate! 🚀

2. Legal Research (Harvey AI - $100M valuation)

Without RAG:

• Lawyer searches case law manually
• Reads dozens of cases
• Writes legal memo
• Time: 8-40 hours
• Cost: $2,000-10,000

With RAG:

• AI searches millions of cases
• Finds relevant precedents
• Generates memo with citations
• Time: 30 minutes
• Cost: $50

Result: 16-80x faster, 40-200x cheaper! ⚖️

3. Medical Documentation (Nuance DAX)

Without RAG:

• Doctor dictates notes
• Transcriptionist types
• Doctor reviews and edits
• Time: 15-30 minutes per patient
• Cost: $50-100 per patient

With RAG:

• AI listens to conversation
• Searches medical guidelines
• Generates clinical notes
• Time: 2 minutes per patient
• Cost: $5 per patient

Result: 7-15x faster, 10-20x cheaper! 🏥

How RAG Works (Simple Explanation):

Step 1: Index Your Documents

Your documents → Embeddings → Vector Database

Example:
"Vacation Policy" → [0.2, 0.8, 0.1, ...] → Stored in Pinecone
"Sick Leave Policy" → [0.3, 0.7, 0.15, ...] → Stored in Pinecone
"Remote Work Policy" → [0.4, 0.6, 0.2, ...] → Stored in Pinecone

Step 2: User Asks Question

User: "How many vacation days do I get?"

Step 3: Retrieve Relevant Docs

Question → Embedding → [0.25, 0.75, 0.12, ...]
Search Vector DB → Find similar vectors
Results:
1. "Vacation Policy" (95% match)
2. "PTO Accrual Rules" (88% match)
3. "Holiday Schedule" (75% match)

Step 4: Generate Answer

Prompt to ChatGPT:
"Context: [Vacation Policy document]
Question: How many vacation days do I get?
Answer based only on the context provided."

ChatGPT: "You receive 15 days of PTO annually..."

Step 5: Return Answer with Sources

Answer: "You receive 15 days of PTO annually..."
Source: Employee Handbook, page 23
Confidence: 95%

RAG vs. Alternatives:

Option 1: Just Use ChatGPT (No RAG)

• ❌ Doesn't know your data
• ❌ Training cutoff (outdated info)
• ❌ Hallucinates (makes stuff up)
• ✅ Fast and cheap

Option 2: Fine-Tune a Model

• ✅ Knows your data
• ❌ Expensive ($1,000-10,000)
• ❌ Can't update easily (need to retrain)
• ❌ Still hallucinates sometimes

Option 3: RAG (Best of Both!)

• ✅ Knows your data
• ✅ Cheap ($10-100/month)
• ✅ Easy to update (just add docs)
• ✅ Minimal hallucinations (grounded in sources)
• ✅ Provides citations

Winner: RAG! 🏆

The Numbers:

Hallucination Rates:

• ChatGPT alone: 15-20% of answers contain errors
• Fine-tuned model: 8-12% errors
• RAG: 2-5% errors (70-90% reduction!)

Cost Comparison (1M queries):

• ChatGPT alone: $2,000
• Fine-tuning: $10,000 (training) + $12,000 (inference) = $22,000
• RAG: $2,000 (LLM) + $500 (vector DB) = $2,500

Update Speed:

• ChatGPT alone: Can't update (training cutoff)
• Fine-tuning: Weeks (need to retrain)
• RAG: Seconds (just add new docs!)

Real Companies Using RAG:

1. Notion AI

• Searches your Notion workspace
• Answers questions about your notes
• Generates content based on your docs

2. GitHub Copilot Chat

• Searches your codebase
• Answers questions about your code
• Suggests changes based on your patterns

3. Perplexity AI

• Searches the web in real-time
• Provides answers with sources
• Always up-to-date

4. You.com

• Combines search + AI
• Cites sources for every claim
• Reduces hallucinations

By the end of this guide, you'll understand:

• How to build a RAG system from scratch
• How to choose the right vector database
• How to optimize retrieval quality
• How to reduce hallucinations by 70-90%
• How to handle large document collections
• How to provide source citations
• Cost optimization strategies

The breakthrough: RAG solved the "knowledge cutoff" problem! Now AI can access ANY information, not just what it was trained on! 🚀

Fun fact: RAG was invented by Facebook AI in 2020, but it exploded in popularity in 2023 when ChatGPT made LLMs mainstream. Now it's the #1 way to build AI applications! 📈

Let's dive in! 🚀

🎨 Visual Understanding

RAG System Architecture

graph TB
    User["User Query:<br/>'What's our vacation policy?'"] --> Embed1[Query Embedding]
    
    Docs["Company Documents"] --> Chunk[Chunk Documents]
    Chunk --> Embed2[Document Embeddings]
    Embed2 --> VectorDB[(Vector Database<br/>Pinecone/Chroma)]
    
    Embed1 --> Search[Semantic Search]
    VectorDB --> Search
    
    Search --> Top["Top-K Results<br/>(3-5 most relevant)"]
    
    Top --> Context[Build Context]
    User --> Context
    
    Context --> Prompt["Augmented Prompt:<br/>Context + Query"]
    
    Prompt --> LLM[Large Language Model<br/>GPT-4/Claude]
    
    LLM --> Response["Generated Answer<br/>+ Citations"]
    
    style User fill:#FF9800,color:#fff
    style VectorDB fill:#2196F3,color:#fff
    style Response fill:#4CAF50,color:#fff

RAG Pipeline Flow

sequenceDiagram
    participant U as User
    participant Q as Query Processor
    participant V as Vector DB
    participant R as Retriever
    participant L as LLM
    
    U->>Q: "What's our vacation policy?"
    Q->>Q: Convert to embedding
    Q->>V: Search similar documents
    V->>R: Return top 5 matches
    R->>R: Rerank by relevance
    R->>L: Context + Query
    L->>L: Generate answer
    L->>U: "You get 15 days PTO...<br/>Source: Employee Handbook p.23"
    
    Note over V,R: Retrieval Phase
    Note over L: Generation Phase

Naive vs Advanced RAG

graph LR
    subgraph "Naive RAG"
        N1[Query] --> N2[Embed]
        N2 --> N3[Search]
        N3 --> N4[Top-K]
        N4 --> N5[Generate]
    end
    
    subgraph "Advanced RAG"
        A1[Query] --> A2[Query Expansion]
        A2 --> A3[Hybrid Search<br/>Vector + Keyword]
        A3 --> A4[Reranking]
        A4 --> A5[Context Compression]
        A5 --> A6[Generate]
        A6 --> A7[Self-Verification]
    end
    
    style N1 fill:#FF9800,color:#fff
    style A1 fill:#4CAF50,color:#fff
    style A7 fill:#2196F3,color:#fff

Document Processing Pipeline

graph TB
    Raw["Raw Documents<br/>(PDF, DOCX, HTML)"] --> Extract[Text Extraction]
    
    Extract --> Clean[Cleaning<br/>Remove headers/footers]
    Clean --> Chunk[Chunking<br/>Split into segments]
    
    Chunk --> Size["Chunk Size:<br/>512-1024 tokens"]
    Size --> Overlap["Overlap:<br/>50-100 tokens"]
    
    Overlap --> Embed[Generate Embeddings]
    Embed --> Meta[Add Metadata<br/>Source, page, date]
    
    Meta --> Store[(Store in<br/>Vector Database)]
    
    style Raw fill:#FF9800,color:#fff
    style Store fill:#4CAF50,color:#fff

Retrieval Strategies Comparison

graph TB
    subgraph "Dense Retrieval (Vector)"
        D1[Semantic Similarity] --> D2[Understands meaning]
        D2 --> D3["'vacation' matches 'PTO'"]
    end
    
    subgraph "Sparse Retrieval (Keyword)"
        S1[Exact Matching] --> S2[Fast & precise]
        S2 --> S3["'vacation' only matches 'vacation'"]
    end
    
    subgraph "Hybrid Retrieval (Best)"
        H1[Combine Both] --> H2[Semantic + Keyword]
        H2 --> H3[Best of both worlds]
    end
    
    D3 --> H1
    S3 --> H1
    
    style D1 fill:#2196F3,color:#fff
    style S1 fill:#FF9800,color:#fff
    style H1 fill:#4CAF50,color:#fff

RAG vs Fine-tuning vs Prompt Engineering

graph LR
    subgraph "Prompt Engineering"
        P1[Add context<br/>to prompt] --> P2[Limited by<br/>context window]
        P2 --> P3[Manual work]
    end
    
    subgraph "Fine-tuning"
        F1[Train model<br/>on data] --> F2[Expensive<br/>$1K-10K]
        F2 --> F3[Hard to update]
    end
    
    subgraph "RAG"
        R1[Retrieve<br/>relevant docs] --> R2[Cheap<br/>$10-100/mo]
        R2 --> R3[Easy to update]
        R3 --> R4[Provides sources]
    end
    
    style P1 fill:#FF9800,color:#fff
    style F1 fill:#F44336,color:#fff
    style R1 fill:#4CAF50,color:#fff

Chunking Strategies

graph TB
    Doc["Long Document<br/>(10,000 words)"] --> Strategy{Chunking Strategy}
    
    Strategy --> Fixed["Fixed Size<br/>512 tokens each"]
    Strategy --> Semantic["Semantic<br/>By paragraphs"]
    Strategy --> Sliding["Sliding Window<br/>With overlap"]
    
    Fixed --> F1["Chunk 1: 0-512"]
    Fixed --> F2["Chunk 2: 512-1024"]
    Fixed --> F3["Chunk 3: 1024-1536"]
    
    Semantic --> S1["Chunk 1: Intro"]
    Semantic --> S2["Chunk 2: Methods"]
    Semantic --> S3["Chunk 3: Results"]
    
    Sliding --> Sl1["Chunk 1: 0-512"]
    Sliding --> Sl2["Chunk 2: 462-974<br/>(50 token overlap)"]
    Sliding --> Sl3["Chunk 3: 924-1436"]
    
    style Doc fill:#FF9800,color:#fff
    style Sliding fill:#4CAF50,color:#fff

🗺️ Concept Mind Map

                RETRIEVAL AUGMENTED GENERATION (RAG)
                              |
        +---------------------+---------------------+
        |                     |                     |
    PIPELINE              COMPONENTS           STRATEGIES
        |                     |                     |
        |                     |                     |
    1. Retrieval          Vector Store         Naive RAG
    - User query          - FAISS              - Simple retrieve
    - Embed query         - Pinecone             & generate
    - Search vectors      - Weaviate           - Fast but basic
    - Get top-k docs      - Chroma
        |                     |                     |
        |                 Embeddings           Advanced RAG
    2. Augmentation       - OpenAI             - Reranking
    - Combine query       - Sentence-BERT      - Query expansion
      with context        - Cohere             - Hybrid search
    - Format prompt       - text-embedding-3   - Better accuracy
    - Add instructions
        |                     |                     |
        |                 LLM Generator        Agentic RAG
    3. Generation         - GPT-4              - Multi-step
    - Send to LLM         - Claude             - Tool use
    - Get response        - Llama              - Self-correction
    - Include citations   - Mistral            - Most powerful
        |
        |
    4. Post-Processing
    - Extract sources
    - Format output
    - Add metadata

Key Workflow:

User Question
     ↓
[Embed Query] → Vector DB Search → Top-K Documents
     ↓
[Augment Prompt] = Question + Context + Instructions
     ↓
[LLM Generation] → Answer with Citations
     ↓
[Post-Process] → Formatted Response

RAG vs Fine-Tuning:

• RAG: Dynamic knowledge, easy updates, no retraining
• Fine-Tuning: Static knowledge, requires retraining, better for style/format

zations](#🎮-interactive-tools--visualizations) 3. Fun Facts & Hacks 4. What You'll Learn 5. Core Concept 6. Code Examples 7. Hands-On Exercises 8. Assignments & Practice Problems 9. Common Pitfalls & Debugging 10. Related Concepts 11. Resources 12. Key Takeaways 13. Next Steps

🛠️ Local Setup (15 Minutes)

Build a complete RAG system locally:

Step 1: Install Dependencies

pip install openai==1.3.0 \
            chromadb==0.4.18 \
            sentence-transformers==2.2.2 \
            langchain==0.1.0 \
            pypdf==3.17.0 \
            python-dotenv==1.0.0

Step 2: Set Up Environment

# Create .env file
echo "OPENAI_API_KEY=your-key-here" > .env

Step 3: Quick RAG Test

from openai import OpenAI
import chromadb
from sentence_transformers import SentenceTransformer
import os
from dotenv import load_dotenv

load_dotenv()

# Initialize
client = OpenAI()
chroma_client = chromadb.Client()
collection = chroma_client.create_collection("knowledge")

# Add knowledge
knowledge = [
    "The Eiffel Tower is 330 meters tall",
    "Paris is the capital of France",
    "The tower was completed in 1889"
]

collection.add(
    documents=knowledge,
    ids=[str(i) for i in range(len(knowledge))]
)

# RAG Query
query = "How tall is the Eiffel Tower?"

# 1. Retrieve relevant docs
results = collection.query(query_texts=[query], n_results=2)
context = "\n".join(results['documents'][0])

# 2. Generate with context
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{
        "role": "user",
        "content": f"Context:\n{context}\n\nQuestion: {query}\nAnswer:"
    }]
)

print(f"Answer: {response.choices[0].message.content}")
print("\n✅ RAG working!")

🧠 Core Concept

The Problem: LLM Limitations

LLMs have three major limitations:

1. Knowledge Cutoff: Training data ends at a specific date
2. Hallucinations: Make up plausible-sounding but false information
3. No Private Data: Can't access your company's documents

The Solution: RAG Pipeline

RAG solves these by retrieving relevant information:

# Traditional LLM (prone to hallucinations)
response = llm("What is our company's vacation policy?")
# LLM guesses: "Probably 2 weeks..." (WRONG!)

# RAG (accurate)
# 1. Retrieve relevant docs
docs = vector_db.search("vacation policy")
# 2. Add to prompt
prompt = f"Context: {docs}\n\nQuestion: What is our company's vacation policy?"
response = llm(prompt)
# LLM answers: "According to the employee handbook, 3 weeks..." (CORRECT!)

RAG Architecture

Basic RAG:

Query → Retrieve → Generate → Answer

Advanced RAG:

Query → Query Expansion → Retrieve → Rerank → Filter → Generate → Verify → Answer

Key Components

1. Document Ingestion:

   • Load documents (PDF, text, web)
   • Chunk into pieces
   • Generate embeddings
   • Store in vector database

2. Retrieval:

   • Convert query to embedding
   • Search vector database
   • Get top K similar chunks

3. Generation:

   • Build prompt with retrieved context
   • Generate answer with LLM
   • Include citations

💻 Code Examples

Example 1: Basic RAG System

from openai import OpenAI
import chromadb
from typing import List

class BasicRAG:
    """
    Simple RAG implementation.
    """
    
    def __init__(self, openai_api_key: str):
        self.llm_client = OpenAI(api_key=openai_api_key)
        self.vector_db = chromadb.Client()
        self.collection = self.vector_db.create_collection("knowledge")
    
    def add_documents(self, documents: List[str]):
        """Add documents to knowledge base."""
        self.collection.add(
            documents=documents,
            ids=[str(i) for i in range(len(documents))]
        )
        print(f"✅ Added {len(documents)} documents")
    
    def retrieve(self, query: str, n_results: int = 3) -> List[str]:
        """Retrieve relevant documents."""
        results = self.collection.query(
            query_texts=[query],
            n_results=n_results
        )
        return results['documents'][0]
    
    def generate(self, query: str, context: List[str]) -> str:
        """Generate answer using retrieved context."""
        # Build prompt
        context_text = "\n\n".join([f"[{i+1}] {doc}" for i, doc in enumerate(context)])
        
        prompt = f"""Answer the question using only the context below. 
If the answer is not in the context, say "I don't have enough information."

Context:
{context_text}

Question: {query}

Answer:"""
        
        # Generate
        response = self.llm_client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0
        )
        
        return response.choices[0].message.content
    
    def query(self, question: str) -> dict:
        """Complete RAG pipeline."""
        # 1. Retrieve
        context = self.retrieve(question)
        
        # 2. Generate
        answer = self.generate(question, context)
        
        return {
            "question": question,
            "answer": answer,
            "sources": context
        }

# Example usage
rag = BasicRAG(openai_api_key="your-key")

# Add knowledge
documents = [
    "The company was founded in 2020 by Jane Smith.",
    "We offer 3 weeks of vacation per year.",
    "The office is located in San Francisco.",
    "We have 50 employees across 3 departments.",
    "Our main product is an AI-powered chatbot."
]

rag.add_documents(documents)

# Query
result = rag.query("How many vacation days do employees get?")

print(f"Question: {result['question']}")
print(f"\nAnswer: {result['answer']}")
print(f"\nSources:")
for i, source in enumerate(result['sources']):
    print(f"[{i+1}] {source}")

Output:

Question: How many vacation days do employees get?

Answer: According to the context, employees get 3 weeks of vacation per year.

Sources:
[1] We offer 3 weeks of vacation per year.
[2] The company was founded in 2020 by Jane Smith.
[3] We have 50 employees across 3 departments.

Example 2: RAG with PDF Documents

from pypdf import PdfReader
import chromadb
from openai import OpenAI
from typing import List
import os

class PDFRagSystem:
    """
    RAG system that works with PDF documents.
    """
    
    def __init__(self, openai_api_key: str):
        self.llm_client = OpenAI(api_key=openai_api_key)
        self.vector_db = chromadb.Client()
        self.collection = self.vector_db.create_collection("pdfs")
    
    def load_pdf(self, pdf_path: str) -> str:
        """Extract text from PDF."""
        reader = PdfReader(pdf_path)
        text = ""
        for page in reader.pages:
            text += page.extract_text() + "\n"
        return text
    
    def chunk_text(self, text: str, chunk_size: int = 1000, overlap: int = 200) -> List[str]:
        """Split text into overlapping chunks."""
        words = text.split()
        chunks = []
        
        for i in range(0, len(words), chunk_size - overlap):
            chunk = ' '.join(words[i:i + chunk_size])
            if chunk:
                chunks.append(chunk)
        
        return chunks
    
    def index_pdf(self, pdf_path: str):
        """Load and index a PDF."""
        # Extract text
        text = self.load_pdf(pdf_path)
        
        # Chunk
        chunks = self.chunk_text(text)
        
        # Add to vector DB
        self.collection.add(
            documents=chunks,
            ids=[f"{os.path.basename(pdf_path)}_{i}" for i in range(len(chunks))],
            metadatas=[{"source": pdf_path, "chunk": i} for i in range(len(chunks))]
        )
        
        print(f"✅ Indexed {pdf_path}: {len(chunks)} chunks")
    
    def query(self, question: str, n_results: int = 3) -> dict:
        """Query the indexed PDFs."""
        # Retrieve
        results = self.collection.query(
            query_texts=[question],
            n_results=n_results
        )
        
        context = results['documents'][0]
        metadatas = results['metadatas'][0]
        
        # Build prompt
        context_text = "\n\n".join([
            f"[Source: {meta['source']}, Chunk: {meta['chunk']}]\n{doc}"
            for doc, meta in zip(context, metadatas)
        ])
        
        prompt = f"""Answer based on the context below. Include source references.

Context:
{context_text}

Question: {question}

Answer:"""
        
        # Generate
        response = self.llm_client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0
        )
        
        return {
            "question": question,
            "answer": response.choices[0].message.content,
            "sources": [{"text": doc, "metadata": meta} 
                       for doc, meta in zip(context, metadatas)]
        }

# Example usage
pdf_rag = PDFRagSystem(openai_api_key="your-key")

# Index PDFs
pdf_rag.index_pdf("company_handbook.pdf")
pdf_rag.index_pdf("product_docs.pdf")

# Query
result = pdf_rag.query("What is the vacation policy?")
print(result['answer'])

Example 3: Advanced RAG with Reranking

from sentence_transformers import SentenceTransformer, CrossEncoder
import chromadb
from openai import OpenAI
from typing import List, Dict
import numpy as np

class AdvancedRAG:
    """
    RAG with reranking for better retrieval quality.
    """
    
    def __init__(self, openai_api_key: str):
        self.llm_client = OpenAI(api_key=openai_api_key)
        self.vector_db = chromadb.Client()
        self.collection = self.vector_db.create_collection("advanced")
        
        # Reranking model (more accurate but slower)
        self.reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
    
    def add_documents(self, documents: List[str], metadatas: List[Dict] = None):
        """Add documents with metadata."""
        self.collection.add(
            documents=documents,
            ids=[str(i) for i in range(len(documents))],
            metadatas=metadatas or [{}] * len(documents)
        )
    
    def retrieve_and_rerank(self, query: str, n_retrieve: int = 10, 
                           n_final: int = 3) -> List[Dict]:
        """
        Two-stage retrieval:
        1. Fast vector search (get top N)
        2. Slow reranking (get top K)
        """
        # Stage 1: Fast retrieval
        results = self.collection.query(
            query_texts=[query],
            n_results=n_retrieve
        )
        
        documents = results['documents'][0]
        metadatas = results['metadatas'][0]
        
        # Stage 2: Rerank with cross-encoder
        pairs = [[query, doc] for doc in documents]
        scores = self.reranker.predict(pairs)
        
        # Sort by reranking score
        ranked_indices = np.argsort(scores)[::-1][:n_final]
        
        return [
            {
                "document": documents[i],
                "metadata": metadatas[i],
                "score": float(scores[i])
            }
            for i in ranked_indices
        ]
    
    def generate_with_citations(self, query: str, context: List[Dict]) -> str:
        """Generate answer with source citations."""
        # Build context with citations
        context_text = "\n\n".join([
            f"[{i+1}] {doc['document']}"
            for i, doc in enumerate(context)
        ])
        
        prompt = f"""Answer the question using the context below. 
Cite sources using [1], [2], etc.

Context:
{context_text}

Question: {query}

Answer (with citations):"""
        
        response = self.llm_client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0
        )
        
        return response.choices[0].message.content
    
    def query(self, question: str) -> Dict:
        """Complete advanced RAG pipeline."""
        # 1. Retrieve and rerank
        context = self.retrieve_and_rerank(question)
        
        # 2. Generate with citations
        answer = self.generate_with_citations(question, context)
        
        return {
            "question": question,
            "answer": answer,
            "sources": context
        }

# Example usage
advanced_rag = AdvancedRAG(openai_api_key="your-key")

# Add documents
documents = [
    "Machine learning is a subset of artificial intelligence.",
    "Deep learning uses neural networks with multiple layers.",
    "Supervised learning requires labeled training data.",
    "Unsupervised learning finds patterns without labels.",
    "Reinforcement learning learns through trial and error."
]

metadatas = [
    {"topic": "ML basics", "difficulty": "beginner"},
    {"topic": "Deep learning", "difficulty": "intermediate"},
    {"topic": "Supervised", "difficulty": "beginner"},
    {"topic": "Unsupervised", "difficulty": "intermediate"},
    {"topic": "RL", "difficulty": "advanced"}
]

advanced_rag.add_documents(documents, metadatas)

# Query
result = advanced_rag.query("What is deep learning?")

print(f"Question: {result['question']}\n")
print(f"Answer: {result['answer']}\n")
print("Sources:")
for i, source in enumerate(result['sources']):
    print(f"[{i+1}] {source['document']}")
    print(f"    Score: {source['score']:.4f}")
    print(f"    Topic: {source['metadata'].get('topic', 'N/A')}\n")

Example 4: Agentic RAG

from openai import OpenAI
import chromadb
from typing import List, Dict, Optional

class AgenticRAG:
    """
    RAG where LLM decides when to retrieve.
    """
    
    def __init__(self, openai_api_key: str):
        self.llm_client = OpenAI(api_key=openai_api_key)
        self.vector_db = chromadb.Client()
        self.collection = self.vector_db.create_collection("agentic")
    
    def add_documents(self, documents: List[str]):
        """Add documents to knowledge base."""
        self.collection.add(
            documents=documents,
            ids=[str(i) for i in range(len(documents))]
        )
    
    def should_retrieve(self, query: str) -> bool:
        """
        Ask LLM if it needs to retrieve information.
        """
        prompt = f"""You are an AI assistant. Determine if you need to search a knowledge base to answer this question.

Question: {query}

Do you need to search the knowledge base? Answer only 'yes' or 'no'.
Answer:"""
        
        response = self.llm_client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=5
        )
        
        answer = response.choices[0].message.content.strip().lower()
        return 'yes' in answer
    
    def retrieve(self, query: str, n_results: int = 3) -> List[str]:
        """Retrieve relevant documents."""
        results = self.collection.query(
            query_texts=[query],
            n_results=n_results
        )
        return results['documents'][0]
    
    def generate(self, query: str, context: Optional[List[str]] = None) -> Dict:
        """Generate answer with or without context."""
        if context:
            context_text = "\n\n".join(context)
            prompt = f"""Answer using the context below.

Context:
{context_text}

Question: {query}

Answer:"""
        else:
            prompt = query
        
        response = self.llm_client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0
        )
        
        return {
            "answer": response.choices[0].message.content,
            "used_retrieval": context is not None
        }
    
    def query(self, question: str) -> Dict:
        """Agentic RAG pipeline."""
        # 1. Decide if retrieval is needed
        needs_retrieval = self.should_retrieve(question)
        
        # 2. Retrieve if needed
        context = None
        if needs_retrieval:
            context = self.retrieve(question)
        
        # 3. Generate
        result = self.generate(question, context)
        
        return {
            "question": question,
            "answer": result["answer"],
            "used_retrieval": result["used_retrieval"],
            "sources": context if context else []
        }

# Example usage
agentic_rag = AgenticRAG(openai_api_key="your-key")

# Add knowledge
documents = [
    "The company was founded in 2020.",
    "We have offices in San Francisco and New York.",
    "Our CEO is Jane Smith."
]

agentic_rag.add_documents(documents)

# Test queries
queries = [
    "What is 2+2?",  # Doesn't need retrieval
    "Who is our CEO?",  # Needs retrieval
    "What is the capital of France?"  # Doesn't need retrieval
]

for query in queries:
    result = agentic_rag.query(query)
    print(f"Q: {query}")
    print(f"A: {result['answer']}")
    print(f"Used retrieval: {result['used_retrieval']}\n")

Example 5: Production RAG System

from openai import OpenAI
import chromadb
from typing import List, Dict, Optional
import logging
from datetime import datetime
import hashlib

class ProductionRAG:
    """
    Production-ready RAG with monitoring and error handling.
    """
    
    def __init__(self, openai_api_key: str, collection_name: str = "production"):
        self.llm_client = OpenAI(api_key=openai_api_key)
        self.vector_db = chromadb.Client()
        self.collection = self.vector_db.get_or_create_collection(collection_name)
        
        # Setup logging
        logging.basicConfig(level=logging.INFO)
        self.logger = logging.getLogger(__name__)
        
        # Metrics
        self.metrics = {
            "total_queries": 0,
            "successful_queries": 0,
            "failed_queries": 0,
            "avg_retrieval_time": 0,
            "avg_generation_time": 0
        }
    
    def add_documents(self, documents: List[str], metadatas: List[Dict] = None) -> bool:
        """Add documents with error handling."""
        try:
            ids = [hashlib.md5(doc.encode()).hexdigest() for doc in documents]
            
            self.collection.add(
                documents=documents,
                ids=ids,
                metadatas=metadatas or [{}] * len(documents)
            )
            
            self.logger.info(f"Added {len(documents)} documents")
            return True
        
        except Exception as e:
            self.logger.error(f"Error adding documents: {e}")
            return False
    
    def retrieve(self, query: str, n_results: int = 3, 
                min_similarity: float = 0.5) -> List[Dict]:
        """Retrieve with similarity threshold."""
        try:
            import time
            start = time.time()
            
            results = self.collection.query(
                query_texts=[query],
                n_results=n_results * 2  # Get more, filter later
            )
            
            # Filter by similarity
            filtered_results = []
            for i, (doc, distance) in enumerate(zip(
                results['documents'][0],
                results['distances'][0]
            )):
                similarity = 1 / (1 + distance)
                if similarity >= min_similarity:
                    filtered_results.append({
                        "document": doc,
                        "similarity": similarity,
                        "metadata": results['metadatas'][0][i] if results['metadatas'] else {}
                    })
            
            retrieval_time = time.time() - start
            self.metrics["avg_retrieval_time"] = (
                self.metrics["avg_retrieval_time"] * self.metrics["total_queries"] + retrieval_time
            ) / (self.metrics["total_queries"] + 1)
            
            return filtered_results[:n_results]
        
        except Exception as e:
            self.logger.error(f"Retrieval error: {e}")
            return []
    
    def generate(self, query: str, context: List[Dict]) -> Optional[str]:
        """Generate with error handling and monitoring."""
        try:
            import time
            start = time.time()
            
            if not context:
                return "I don't have enough information to answer this question."
            
            # Build prompt
            context_text = "\n\n".join([
                f"[{i+1}] {doc['document']} (Relevance: {doc['similarity']:.2f})"
                for i, doc in enumerate(context)
            ])
            
            prompt = f"""Answer the question using only the context below.
If the context doesn't contain the answer, say "I don't have enough information."
Include source citations [1], [2], etc.

Context:
{context_text}

Question: {query}

Answer:"""
            
            response = self.llm_client.chat.completions.create(
                model="gpt-3.5-turbo",
                messages=[{"role": "user", "content": prompt}],
                temperature=0
            )
            
            generation_time = time.time() - start
            self.metrics["avg_generation_time"] = (
                self.metrics["avg_generation_time"] * self.metrics["total_queries"] + generation_time
            ) / (self.metrics["total_queries"] + 1)
            
            return response.choices[0].message.content
        
        except Exception as e:
            self.logger.error(f"Generation error: {e}")
            return None
    
    def query(self, question: str) -> Dict:
        """Complete RAG pipeline with monitoring."""
        self.metrics["total_queries"] += 1
        
        try:
            # Retrieve
            context = self.retrieve(question)
            
            # Generate
            answer = self.generate(question, context)
            
            if answer:
                self.metrics["successful_queries"] += 1
                
                return {
                    "question": question,
                    "answer": answer,
                    "sources": context,
                    "timestamp": datetime.now().isoformat(),
                    "success": True
                }
            else:
                self.metrics["failed_queries"] += 1
                return {
                    "question": question,
                    "error": "Generation failed",
                    "success": False
                }
        
        except Exception as e:
            self.metrics["failed_queries"] += 1
            self.logger.error(f"Query error: {e}")
            return {
                "question": question,
                "error": str(e),
                "success": False
            }
    
    def get_metrics(self) -> Dict:
        """Get system metrics."""
        return {
            **self.metrics,
            "success_rate": (
                self.metrics["successful_queries"] / max(self.metrics["total_queries"], 1)
            ) * 100
        }

# Example usage
prod_rag = ProductionRAG(openai_api_key="your-key")

# Add documents
documents = [
    "Our product is an AI chatbot for customer service.",
    "We support integration with Slack, Teams, and Discord.",
    "Pricing starts at $99/month for up to 1000 conversations."
]

prod_rag.add_documents(documents)

# Query
result = prod_rag.query("What integrations do you support?")

if result["success"]:
    print(f"Answer: {result['answer']}\n")
    print("Sources:")
    for source in result['sources']:
        print(f"- {source['document'][:80]}... (Relevance: {source['similarity']:.2f})")

# Check metrics
print("\n=== System Metrics ===")
metrics = prod_rag.get_metrics()
for key, value in metrics.items():
    print(f"{key}: {value}")

📝 Assignments & Practice Problems

Assignment 1: Document QA System ⭐

Difficulty: Easy | Time: 45 minutes

Build a system that answers questions about your documents.

Requirements:

• Load text files
• Chunk and index
• Answer questions
• Show sources

Assignment 2: Multi-Document RAG ⭐⭐

Difficulty: Medium | Time: 90 minutes

Create RAG system handling multiple document types.

Requirements:

• Support PDF, TXT, MD files
• Metadata tracking
• Source attribution
• Relevance filtering

Assignment 3: Production RAG Pipeline ⭐⭐⭐

Difficulty: Hard | Time: 120 minutes

Build production-ready RAG system.

Requirements:

• Error handling
• Monitoring and metrics
• Caching
• Reranking
• API interface
• Unit tests

❌ Bad: No relevance filtering

context = retrieve(query, n=5)

✅ Good: Filter by similarity

context = [doc for doc in retrieve(query, n=10) if doc['similarity'] > 0.7]

### Pitfall 2: Too Much Context

```python
# ❌ Bad: 10 long chunks (exceeds context window)
context = retrieve(query, n=10)

# ✅ Good: 3 relevant chunks
context = retrieve(query, n=3)

Pitfall 3: No Source Citations

# ❌ Bad: No way to verify
answer = generate(query, context)

# ✅ Good: Include citations
answer = generate_with_citations(query, context)

pip install langchain      # RAG framework
pip install llama-index    # Data framework for LLMs
pip install haystack-ai    # Production RAG