How does an LLM generate a response? It receives your question alongside a system prompt and special markers, breaks them into tokens and vectors, transforms those vectors through repeated attention and feed-forward layers, and repeatedly predicts what token should come next until the turn is complete.
The system instruction and user question are joined with special role markers.
These vectors pass through many layers. In each layer, the attention block allows token representations to interact and gather context (e.g., “France” gathering information from “capital”), while the feed-forward layer transforms each token independently and in parallel.
Self-attention computes how much each token should draw information from earlier positions in the context window. “France” gathers information from “capital” to build a richer representation.
Finally, the model uses the representation at the end of the sequence to produce a probability distribution. Here, it gives ~75% to “The” and ~19% to “Paris”. If it chooses “The”, it adds it to the sequence and runs the same process again.
Tokens are mapped to vectors in a continuous multi-dimensional space. Words with related functions or topics form semantic clusters. Drag to rotate in 3D, select words, and inspect their numerical vector dimensions.