What is Training Data? Why does it exist? Where does it appear in Pretraining?
Draw or describe a visual model of Training Data. What does it look like conceptually?
Work through a concrete manual example of Training Data with real numbers or pseudocode.
Implement Training Data from scratch in Python. Keep it minimal — just enough to see it work.
How is Training Data different from related concepts in Pretraining? What are the trade-offs?
Explain Training Data in plain language using an analogy. Could a non-expert follow it?
When you start experimenting, these are the places to get real data — from classic tabular datasets for understanding fundamentals, all the way up to LLM-scale web corpora:
train.py
📋 full plan →