an interactive explainer
One word at a time
How a language model writes: it looks at the text so far, gives every possible next word a chance, picks one — and repeats. Step through a real example, word by word.
What could come next?
the model's pick: The
temperature
1.00
That's the whole sentence — fourteen choices, each one a probability. Press Replay and try a different temperature.
word 1 of 14
p(w) = es(w) / T ÷ Σv es(v) / T
Each candidate word w carries a raw score s(w) — engineers call it a logit. Dividing by the temperature T and applying the softmax turns those scores into probabilities that add up to exactly 1. Low T sharpens the odds toward the favorite; high T flattens them and gives wilder words a real shot.
| word | score s | es/T | probability |
|---|
Real models usually trim the list first — keeping only the top k candidates (top-k) or cutting where the cumulative probability reaches about 0.9 (top-p). We keep just five so you can see every bar.