Why answers differ

Ask the same assistant the same question twice and you will often get two different answers — different words, different structure, sometimes a different recommendation. For software, this is bizarre: your calculator does not sometimes feel that 7 × 8 is 54. People sense something loose in the machine, and they're right. This short lesson names the looseness, shows the dial that controls it, and turns it from unsettling quirk into a tool you deliberately use.

Prediction is a ranking, not an answer

Go back to the loop from lesson 1, one level sharper. At each step the model doesn't produce the next token — it produces a score for every token on the menu: a ranked list of maybe a hundred thousand candidates. Continuing "The parcel doesn't fit in the ___", the list might effectively say: "van" very likely, "truck" likely, "letterbox" plausible, "courier" a stretch, "photosynthesis" astronomically unlikely. That ranked list is the model's entire opinion. Something still has to pick one token from it — and that choice, repeated every single word, is where the variety enters.

The dial between reliable and interesting

The obvious strategy — always take the top-ranked token — is called greedy selection, and it produces the same answer every time. It also produces noticeably flat, repetitive, safest-word-at-every-turn text, and it can steer weirdly: like a writer who never once picks the second-best word, it forecloses every interesting path. So instead, systems sample: mostly take high-ranked tokens, occasionally a respectable runner-up. How adventurous that sampling is, is set by a dial famously called temperature. Cold: stick to the top of the rankings — repeatable, conservative, a touch dull. Warm: give runners-up a real chance — varied, livelier, and occasionally odd. Chat assistants run moderately warm by default, which is exactly why the same question yields different answers on different runs: at a hundred forks, the dice landed differently, and each choice fed back into the loop to shape every word after it.

One immediate practical payoff: regeneration is not a gimmick. The "try again" button re-rolls those dice, and the second draft is not a copy with synonyms — it's a genuinely different path through the rankings. For anything creative — subject lines, names, framings, openings — asking for five variants is asking the machine to do the one thing randomness is for. Conversely, when you need consistency (the same classification for the same input, every time), you now know why builders turn the temperature down to zero in their settings — and why your chat answers will never be perfectly repeatable at the defaults.

Randomness is not the error source

A misconception to retire before it settles: temperature is not why models hallucinate. A hallucination (lesson 7) is the top-ranked continuation being wrong — turn the dial to zero and the model states the same falsehood with the same serenity, every single time. Randomness decides which plausible path gets walked; it doesn't decide what counts as plausible. Blaming the dice for hallucination is blaming the shuffle for the deck's contents.

There's also a quieter insight here about what "the model's answer" even means. There is no single answer-in-waiting; there is a distribution — a weighted space of answers it might give, from which each run draws one. Once you see that, the mature habit follows naturally: for anything that matters, one sample is an anecdote. Ask twice. If the recommendation flips between runs, you've learned the model is genuinely uncertain there — which is real information, and information the confident tone of any single answer would never have shown you.

Try this in your chatbot: ask "Name a good title for a logistics newsletter" five times in five fresh chats — count the distinct answers. Then ask "What is 7 × 8?" five times. You've just mapped where the machine's rankings are flat (many good continuations, dice matter) versus spiked (one dominant continuation, dice irrelevant) — which is precisely the shape of when to trust one run.