From autocomplete to assistant

There's a gap in the story so far, and you may have noticed it. A pretrained model is a next-text-predictor — so if you hand it "Write me a leave application", the truly likely continuation on the internet is… more of someone's question, or a forum reply saying "just google it lol". Raw pretrained models really do behave like this: brilliant, erratic, and utterly unhelpful — a parrot of the whole internet, including the internet's worst days. Something turns that into the polite, structured assistant you actually meet. This lesson is that something, and it explains more of the assistant's personality than any other part of the machine.

Step one: show it what an assistant looks like

The first fix is instruction tuning. Take the pretrained model and continue its training — briefly, compared to pretraining — on a much smaller, hand-built dataset of a specific shape: instruction in, excellent answer out. "Summarize this article" followed by an actual good summary; "Explain GST to a shopkeeper" followed by a clear, structured explanation; tens of thousands of these, written or curated by people. The model already knows nearly everything it needs (that's the blur from lesson 5); this step teaches it the format of being helpful — that its job on seeing a request is to fulfil it, structured and complete, not to continue the forum thread. The knowledge was there; the persona is installed here.

Step two: teach it what people prefer

Instruction tuning alone still leaves rough edges — answers that are technically responsive but rambling, tone-deaf, or quietly harmful. The polish comes from preference training: generate two candidate answers to the same request, have humans (and increasingly, AI judges following human-written guidelines) pick the better one, and nudge the model toward the winning style — hundreds of thousands of times. You may have met this stage personally: every time a chatbot shows you two answers and asks "which is better?", you are a volunteer preference-trainer. The best-known technique here goes by RLHF — reinforcement learning from human feedback — a name worth recognising in articles even though the acronym matters less than the idea: the assistant's manner is a trained artifact, tuned to please human judges.

Read that sentence again, because both halves cut. It is why the assistant is patient, structured, and endlessly civil at 2 a.m. — those styles win preference votes. And it is why the assistant is agreeable to a fault: judges, being human, systematically favour answers that flatter their premise over answers that open with "actually, you're mistaken". Trained sycophancy is the result — ask "why is my plan brilliant?" and the machine optimised for your approval will find brilliance. The practical antidote is deliberately inviting the pushback the training suppressed: "argue against this", "what would a critic say?", "steelman the other side". The vote-pleaser will please that framing just as diligently — you're not fighting sycophancy so much as steering it.

The rulebook on top

One more layer and the assistant is assembled. Providers give their models standing rules — some trained in as values during preference training, some literally written as instructions the model reads before every conversation (the industry calls that a system prompt). Refusing to help with weapons, declining medical diagnoses, the tone taken on sensitive topics, even the assistant's name: rulebook, not physics. Different companies write different rulebooks, which is a large part of why Claude, ChatGPT and Gemini feel different even though all three are transformers full of attention layers doing next-token prediction. When one assistant declines and another obliges, you are usually watching two different policy documents, not two different capabilities.

It's worth assembling the full creature once, plainly: a next-word predictor (lesson 1), reading token IDs (lesson 2) mapped into meaning-space (lesson 3), contextualised by attention (lesson 4), loaded with a frozen compressed library (lesson 5), wearing a trained persona tuned to human approval, under a corporate rulebook. That is what you're talking to. No part of it is magic; every part explains behaviour you've already seen.

Try this in your chatbot: describe any decision you're leaning toward and ask "Why is this a good idea?" — then, fresh chat, same decision, "Why is this a mistake?" Watch the same machine argue both sides with equal conviction. That's preference training showing you its seams — and your permanent reminder to never poll it with a leading question you care about.