🚕
ERA A · HUMAN EXPERIENCE
The knowledgeable driver
The driver learns streets, landmarks, shortcuts and routines through experience. Useful knowledge lives in the driver's memory; coverage depends on what they have encountered.
A field guide to the AI era · Orientation 00.5
From memorized roads
to machine intelligence
Imagine needing a driver who has memorized every road in a city. Then digital maps arrive: route knowledge becomes available on demand. Now imagine an assistant that can explain, write, translate, code and help solve problems. How did we get here—and what is really happening inside it?
🧭 Story before equations🗓️ Historical milestones🔬 Interactive mental models⏱ 30–45 minutes
THE BIG SHIFT · WATCH THE STORY
From remembering the world to working with a model
Core idea: capabilities emerge from a system, not magic.AI can help—but it does not guarantee the destination is correct.
01 / Start with something familiar
The driver, the map and the AI assistant
A good analogy makes an unfamiliar idea approachable. It also has limits. Use the driving story to understand the shift from relying on knowledge held by one person to systems that learn patterns, retrieve information and assist with decisions.
🗺️
ERA B · EXTERNAL INFORMATION
The driver with digital maps
A navigation system stores map data and can calculate routes. GPS and traffic feeds add live signals. The driver still has to interpret conditions and act safely.
🧠
ERA C · LEARNED LANGUAGE MODEL
The AI assistant
An LLM learns patterns from huge training corpora and can generate answers across many domains. It is not simply a live encyclopedia: it can be wrong, stale or confidently incomplete.
A useful mental model: an LLM is like a remarkably well-read assistant—not an all-knowing, infallible wisdom master.
Where the analogy breaks: a map usually retrieves structured map data and computes a route. An LLM typically generates text from learned parameters and the current context. Give it search, databases, tools and verification, and the surrounding system can become much more capable.
02 / Why AI began
What if we could build systems that learn?
Traditional software works by executing rules written by people. That remains essential. But perception, language and messy real-world tasks are hard to capture with a complete list of hand-written rules. AI research explored ways to represent knowledge, search possibilities, learn from examples and make predictions.
PROGRAMMED RULES
“Tell me exactly how.”
Useful when steps and rules can be specified. Brittle when the world presents exceptions the programmer did not anticipate.
LEARNING FROM DATA
“Show me examples.”
A model adjusts internal parameters to capture statistical regularities. It can generalize to new examples, but may also learn unwanted biases or shortcuts.
SYSTEMS THAT ASSIST
“Help me do the task.”
Modern products combine models with instructions, retrieval, tools, interfaces and safeguards. The model is one important component of the system.
03 / The milestones
A long road—not a sudden invention
Select a milestone to explore what changed, why it mattered, and what limitation remained. Dates mark influential milestones, not the only work happening in that period.
INTERACTIVE HISTORY · SCRUB THROUGH THE IDEAS
AI did not suddenly appear. The ingredients accumulated.
Move through the eras. Watch the dominant approach change—and see the bottleneck that pushed research toward the next idea.
1956
THE BIG IDEA
Represent knowledge with symbols and rules
Early AI research explored whether reasoning could be expressed as symbolic operations and search through possibilities.
THE BOTTLENECK
The world has too many exceptions
This helped motivate approaches that could learn patterns from examples rather than require every rule to be written by hand.
The animated artwork is a conceptual illustration of the dominant idea in each era—not a historical photograph, a benchmark, or a literal model architecture.
1940s–1950s
Can a machine reason?
The foundations of computation and early debates about machine intelligence set the stage.
From philosophical question to research program
The important shift was making questions about intelligence precise enough to investigate computationally.
What this era could—and couldn't—do
Why the next step mattered:
1940s–50s
Foundations of computation and machine intelligence
Formal models of computation, early neural-network ideas and Turing's 1950 discussion of machine intelligence helped establish the questions and tools for the field.
Foundations1956
Dartmouth workshop: AI becomes a named research agenda
The Dartmouth summer research project helped establish “artificial intelligence” as a field. Early optimism focused on symbolic reasoning, search and representing knowledge.
A field takes shape1960s–70s
Symbolic AI and early neural networks
Researchers built systems around explicit symbols, rules and search. Perceptrons and other neural ideas explored learning from examples, but hardware, data and algorithmic limits constrained practical reach.
Rules and representations1970s–90s
AI winters and expert systems
Periods of reduced funding followed overpromising and disappointing results. Expert systems delivered value in narrow domains, but rule maintenance and brittleness made broad scaling difficult.
Limits become visible1986 onward
Backpropagation renews interest in neural learning
Work popularizing backpropagation made it practical to train multi-layer networks by propagating learning signals backward through differentiable computations. It was important, but not enough by itself to produce today's systems.
Learning parameters1990s–2000s
Statistical machine learning grows
Probabilistic and data-driven methods became influential in speech, search, vision and language. Instead of hand-writing every rule, systems estimated patterns from examples.
Data-driven prediction2006–2012
Deep learning gains momentum
Improved training techniques, larger datasets and GPU computing helped deeper neural networks learn useful representations. The 2012 AlexNet result on ImageNet became a visible milestone for deep learning in computer vision.
Data + compute + methods2014–2016
Embeddings, sequence models and attention develop
Learned word representations, recurrent networks, encoder–decoder systems and attention advanced how models represented language and handled sequences. These ideas helped prepare the ground for Transformers.
Representations and context2017
“Attention Is All You Need” introduces the Transformer
The Transformer made attention the core of a sequence architecture and enabled more parallelizable training than recurrent approaches. It became a key foundation for many modern language models.
A new architecture2018–2020
BERT, GPT and large-scale pretraining
BERT showed the impact of bidirectional contextual pretraining for language understanding. GPT-style decoder models scaled autoregressive next-token prediction. Large pretrained models could be adapted to many tasks.
Foundation models2020–2022
Scale, prompting and instruction following
Large models showed broader few-shot behaviour; instruction tuning and human-feedback approaches helped make them more useful in dialogue. ChatGPT's public release in 2022 made conversational AI widely accessible.
AI enters everyday workflows2023 onward
Multimodal models, tools and agentic systems
AI systems increasingly work across text, images, audio and video, and can use external tools. Research continues on reliability, evaluation, reasoning, efficiency, safety and grounded action.
The current frontierHistorical lesson: each wave inherited ideas from earlier waves and ran into new limits. Progress was uneven. Today's AI is the result of decades of theory, experiments, infrastructure, data and engineering—not one breakthrough alone.
04 / Why the current era took off
Three forces reinforced each other
No single ingredient explains modern AI. The breakthrough came from the interaction of more data, more practical compute and better methods.
EXPLORE THE ENGINEERING FLYWHEEL
Why the modern AI era accelerated
Move the controls and watch the illustrative capacity indicator change. This is a conceptual model—not measured performance, not a benchmark, and not a claim that “more is always better.”
Illustrative system potential: 62/100Quality, data rights, evaluation and deployment still constrain outcomes.
The glowing paths illustrate information flowing through a neural network. They do not display a real model’s weights or training metrics.
🗃️
Data
Large digital collections provide examples of language, code, images and other patterns. Quality, coverage, privacy and rights still matter.
⚡
Compute
GPUs and distributed infrastructure made large-scale numerical training feasible. Scale also brings cost, energy and deployment trade-offs.
🧩
Methods
Backpropagation, optimization, deep networks, attention, Transformers and post-training made learning and using larger models more effective.
DATAMore examples
COMPUTEMore operations
METHODSBetter learning
PRETRAINLearn patterns
POST-TRAINShape responses
05 / The wise assistant idea
What an LLM learns—and what it doesn't guarantee
Think: a remarkably well-read assistant
During training, the model adjusts a large set of parameters to predict tokens from context. Those parameters can encode useful facts, concepts, relationships and language patterns. The model may generate a helpful answer without retrieving a stored paragraph.
But “trained on lots of data” does not mean “knows everything.” Coverage is uneven; information may be outdated; a fluent answer can be false.
Knowledge is not the same as reliable action
- Knowledge: learned patterns in parameters or retrieved information.
- Reasoning: working through a problem or comparing possibilities.
- Current facts: search, databases, sensors or live sources.
- Action: calling APIs, running code or using tools.
- Verification: testing claims against evidence or outcomes.
06 / Training versus using the model
How the learned system answers you
Switch between the two modes. Training changes the model's parameters. Inference uses the trained parameters to generate an answer.
Training: learn patterns from examples
The model predicts training targets, computes a loss and uses optimization to adjust parameters. This repeats over many examples.
Example → prediction → error → update
Training does not mean a person manually enters every fact as a database row.
The simplified generation loop
01Text → tokens
02Token IDs → vectors
03Transformer → scores
04Select next token
Then the process repeats, appending tokens until generation stops. Tools and retrieval may add external information to a larger assistant system.
07 / Quick checkpoint
Which explanation is strongest?
Why did modern AI accelerate in recent years?
08 / Back to the roadmap
Now open the engine, one part at a time
This chapter gives the motivation, history and system-level mental model. The next sections provide the technical depth; you do not need to master every historical detail before continuing.
00 · FOUNDATIONEssential Math & Intuitive Foundations
00.5 · ORIENTATIONMeet AI: History and Big Picture
01 · REPRESENTATIONTokens → vectors → embeddings
02–03 · CONTEXTAttention, Transformers and efficient context
NEXT STEP
Understand Language
Start with tokens: how text is segmented and mapped to IDs.
DEEPER SYSTEM VIEW
Anatomy of AI
Use your existing inference-pipeline page for a more technical system walkthrough.
BUILD ALONGSIDE
MiniLLM Lab
Turn each new concept into a small experiment, then gradually assemble a tiny language model.
The journey: understand why the field evolved → see the whole system → learn its mechanisms → implement them → evaluate what works and where it fails.