What is Inference? Why does it exist? Where does it appear in LLMs?
Draw or describe a visual model of Inference. What does it look like conceptually?
How is Inference different from related concepts in LLMs? What are the trade-offs?
Explain Inference in plain language using an analogy. Could a non-expert follow it?
1. Training Phase (Studying at home): A student practices math problems like 5 + 2 = 7. As he studies, he makes mistakes, corrects his thinking, and gradually internalizes the underlying logic of addition. In an LLM, this is backpropagation adjusting the model's weights.
2. Inference Phase (The Exam Hall): On exam day, he is presented with a brand new question he never saw in his practice book: 9 + 7 = ?. Because his brain mastered the underlying logic, he accurately solves it as 16 without having memorized that exact combination.
⚡ Key Takeaway: During Inference, the model's weights are completely frozen. The LLM does not merely recite memorized text; it uses its learned mathematical relationships to generate context-aware predictions for novel prompts it has never seen before.