BeginnerEstimated time: 45 mins

Chat with PDF

Build an interactive document question-answering app using Python, LangChain, ChromaDB, and Streamlit.

Tech Stack:Python 3.10+LangChainChromaDBStreamlitOpenAI API / Local LLM

System Architecture

PDF Upload ➔ PyPDFLoader ➔ RecursiveCharacterTextSplitter ➔ OpenAI / HuggingFace Embeddings ➔ ChromaDB Vector Store ➔ RetrievalQA Chain ➔ Streamlit Chat UI

Prerequisites

  • Python 3.10 or higher installed
  • Basic knowledge of Python virtual environments
  • OpenAI API key (or local Ollama instance)

Installation Commands

terminalbash
pip install streamlit langchain langchain-community langchain-openai chromadb pypdf python-dotenv

Folder Structure

Directory Layouttext
chat-with-pdf/
├── app.py              # Main Streamlit application
├── requirements.txt    # Project dependencies
├── .env.example        # Environment variable template
└── README.md           # Setup and usage instructions

Step-by-Step Implementation

1

Environment Configuration and Imports

Set up your API keys using python-dotenv and import the necessary modules from Streamlit, LangChain, and Chroma.

app.py (Part 1)python
import os
import streamlit as st
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
import tempfile

load_dotenv()
2

Document Ingestion and Chunking

Create a function to parse the uploaded PDF and split it into chunks of 1000 characters with 200 character overlap.

app.py (Part 2)python
def process_pdf(uploaded_file):
    with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp_file:
        tmp_file.write(uploaded_file.read())
        tmp_path = tmp_file.name

    loader = PyPDFLoader(tmp_path)
    docs = loader.load()
    os.remove(tmp_path)

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
        length_function=len
    )
    chunks = text_splitter.split_documents(docs)
    return chunks
3

Vector Store & Embeddings Setup

Embed the document chunks and index them into an in-memory ChromaDB vector database.

app.py (Part 3)python
def get_vectorstore(chunks):
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    vectorstore = Chroma.from_documents(chunks, embeddings)
    return vectorstore
4

RAG Retrieval Chain and Streamlit UI

Assemble the conversational prompt, connect the retriever with ChatOpenAI, and render a reactive chat interface.

app.py (Part 4)python
st.set_page_config(page_title="Chat with PDF", page_icon="📄")
st.title("📄 Chat with PDF using RAG")

uploaded_file = st.sidebar.file_uploader("Upload PDF", type="pdf")

if uploaded_file:
    if "vectorstore" not in st.session_state:
        with st.spinner("Processing PDF..."):
            chunks = process_pdf(uploaded_file)
            st.session_state.vectorstore = get_vectorstore(chunks)
            st.sidebar.success(f"Indexed {len(chunks)} chunks!")

    system_prompt = (
        "You are an assistant for question-answering tasks. "
        "Use the following pieces of retrieved context to answer the question. "
        "If you do not know the answer, say that you don't know.\n\n{context}"
    )
    prompt = ChatPromptTemplate.from_messages([
        ("system", system_prompt),
        ("human", "{input}"),
    ])

    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    retriever = st.session_state.vectorstore.as_retriever(search_kwargs={"k": 3})
    question_answer_chain = create_stuff_documents_chain(llm, prompt)
    rag_chain = create_retrieval_chain(retriever, question_answer_chain)

    user_query = st.chat_input("Ask a question about your document:")
    if user_query:
        st.chat_message("user").write(user_query)
        response = rag_chain.invoke({"input": user_query})
        
        with st.chat_message("assistant"):
            st.write(response["answer"])
            with st.expander("View Retrieved Sources"):
                for idx, doc in enumerate(response["context"]):
                    st.caption(f"**Chunk {idx+1} (Page {doc.metadata.get('page', 0) + 1}):**")
                    st.text(doc.page_content[:300] + "...")

Example Execution (Input & Output)

Input Query:
What are the three main types of cloud computing services discussed in Chapter 2?
Generated Grounded Output:
Based on the uploaded document (Chapter 2, Page 14), the three primary cloud service models are: 1. Infrastructure as a Service (IaaS) — provides virtualized computing resources. 2. Platform as a Service (PaaS) — delivers hardware and software tools over the internet. 3. Software as a Service (SaaS) — provides full software applications managed by a third party.

Why this works: The retriever extracted chunks from Page 14 with high cosine similarity, and the LLM synthesized the grounded answer with citations.

Troubleshooting Common Gotchas

OpenAI RateLimitError or quota exceeded

Verify your API credits or switch OpenAIEmbeddings to HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2") for 100% free local embeddings.

Empty text returned from PyPDFLoader on scanned PDFs

PyPDF extracts digital text layers. For scanned image-based PDFs, integrate OCR via pdf2image and pytesseract or use UnstructuredPDFLoader.

Known Limitations

  • •Single-session memory by default (in-memory ChromaDB resets on server restart).
  • •Does not extract complex tables or embedded charts into structured Markdown.

Security Considerations

  • •Never commit hardcoded API keys; always use .env files with .gitignore.
  • •Sanitize uploaded file paths to prevent directory traversal vulnerabilities.
Back to all projects