Build Your Own LLM
Instead of learning AI concepts individually and connecting them later — choose one meaningful project, build it progressively, and learn every concept when you need it. You won't just know how an LLM works. You'll understand why each component exists because you'll have built it yourself.
The learning loop — apply it to every single topic
The 7-Stage Build
Each stage adds a new capability to MiniLLM Lab. The project evolves as your knowledge grows — nothing is built before it's understood.
Text → Numbers
Before a model can do anything, it must convert raw text into a form it can process.
Learn these topics
What you'll understand
- Why tokens, not characters or words
- How BPE merges produce a vocabulary
- Why embeddings are learned, not programmed
- The distributional hypothesis
- Sparse vs dense vectors
Sequential Understanding
A single vector per token is not enough — words change meaning based on context. This stage teaches the model to look at sequences.
Learn these topics
What you'll understand
- Why RNNs fail on long sequences
- How LSTMs extend memory with gates
- Why attention fixes the bottleneck
- How "bank" gets different vectors in context
The Architecture
Combine everything learned so far — attention, embeddings, position — into the architecture that powers all modern LLMs.
Learn these topics
What you'll understand
- Why Q/K/V decomposition works
- How multiple attention heads see different patterns
- Why position must be injected
- How decoder-only vs encoder-decoder differ
Learning from Data
A model with random weights knows nothing. Training is the process that gives the weights meaning through exposure to data.
Learn these topics
- Self-supervised pretraining
- Loss functions & cross-entropy
- Backpropagation & gradients
- Optimizers (Adam, SGD)
- Fine-tuning
What you'll understand
- How next-token prediction teaches language
- Why loss curves look the way they do
- What a gradient actually represents
- When to freeze vs fine-tune embeddings
Inference
Training is done. Now: how do you take a trained model and get it to produce useful responses from a prompt?
Learn these topics
- Prompts & instruction formats
- Context windows
- Token generation & autoregressive decoding
- Temperature, top-p, top-k sampling
- Stopping criteria & EOS tokens
- LLM text generation
What you'll understand
- What happens token-by-token during inference
- Why temperature changes "creativity"
- Why context length is a hard limit
- The difference between greedy, beam, and sampling
Retrieval-Augmented Generation
Your tiny model can't memorize everything. But it doesn't have to — attach a retrieval layer and it can look things up at inference time.
Learn these topics
What you'll understand
- Why RAG beats fine-tuning for factual recall
- How chunk size affects retrieval quality
- Why cosine similarity works as a relevance signal
- How to evaluate a RAG system (precision, faithfulness)
Tools, Agents & Production
The final stage: your model stops being a demo and becomes something that retrieves, reasons, calls tools, and produces verifiable answers.
Learn these topics
What you'll understand
- The agent loop: perceive → plan → act → observe
- How tool schemas are declared and called
- Why quantization reduces memory without killing quality
- How to evaluate agentic behavior
🏁 The Destination: Two Layers, One Project
At the end of MiniLLM Lab you'll have built two things. The first teaches you AI from the inside out. The second proves the knowledge is real by solving an actual user problem.
🔬 Learning engine
Your tiny language model — inspectable internals, hand-coded components, visible math. This is where you understand why every piece of the architecture exists.
🚀 Real-world product
A personal knowledge assistant that answers questions from your own documents. RAG + tool use + a pretrained model for generation. This proves the learning is applicable, not just conceptual.