Interactive explainer · no real AI inside

How a language model writes, one word at a time

An AI never writes a whole sentence at once. It looks at everything so far, gives every possible next word a probability, picks one — then asks the same question again. Step through a real example below.

The text so far Step 0 of 8

Tip: the ← and → arrow keys also step through the sentence.

What comes next? sum = 100%

Temperature — how creative is the pick? 1.00

Before any picking happens, the model gives every candidate word a raw score, called a logit (z). A bigger logit means the model likes that word more. These raw scores are what the bars start from.

To turn logits into probabilities, the scores are squashed through the softmax function, with a temperature T:

p(word_i) = e^{z_i / T} / Σ_j e^{z_j / T}

Dividing z by T is the whole trick: a small T stretches the differences apart, so the favourite dominates. A large T squashes them together, so nearly everything becomes equally likely. Try T = 0.3 versus T = 2.0 above and watch the bars.

Real systems also use top-k (keep only the k highest-scoring words) or top-p, and throw everything else away before sampling. With top-k = 1 the model always takes the winner — no creativity at all.

word	logit z	z ÷ T	softmax p