How AI Actually Works

Lesson 8 of 14

AI's working memory

Every long chat eventually hits a strange moment: the assistant that knew your name, your project, and your constraints… gently loses the thread. It re-asks something you settled an hour ago, or its answers drift generic. People read this as moodiness. It is arithmetic. The model has a hard limit on how much conversation it can see at once — the context window — and understanding it explains the drift, the fix, and (next lesson's bonus) a dial that changes how the model sounds.

The desk, not the diary

Here is the mechanical truth, and it surprises almost everyone: the model remembers nothing between turns. Not between chats — between messages in the same chat. Each time you hit send, the system gathers the conversation so far — every message of yours, every reply of its — and feeds the whole transcript through the model again, from scratch, to predict the next reply. The illusion of a companion who's been listening is re-manufactured on every turn by re-reading the entire exchange. (Add the rulebook from lesson 6 and any memory-profile text your assistant keeps: those ride along at the top of the bundle, which is how they work — they're simply text the model re-reads every time.)

The context window is the size limit on that bundle, measured — of course — in tokens (lesson 2 pays off again). As of mid-2026, frontier models carry windows from a couple of hundred thousand tokens up to around a million: several novels' worth, absurdly beyond 2022's few pages. But three realities keep the limit relevant despite the size. Files count — "chat with your PDF" means the PDF (or retrieved chunks of it; lesson 10) occupies the desk alongside the chat. Overflow evicts — when a marathon chat outgrows the window, older middles get dropped or auto-summarised, and the details in them are simply gone from view, which is exactly the moment the assistant re-asks the thing you told it. And attention thins — even within the window, very long contexts get blurrier treatment in the middle stretches; the model handles openings and endings of its desk better than page sixty of one-twenty. A vast desk, unevenly lit.

The context window: a desk that holds the whole conversation — until it doesn't
The context window: a desk that holds the whole conversation — until it doesn't

Working with the desk

Three habits fall straight out of the mechanics. Start fresh often. A new chat isn't losing a relationship — it's a clean desk. For a new task, fresh context beats a two-hundred-message chat carrying fossil instructions that still steer replies. (The everyday-work course builds standing context properly — projects and instructions — precisely so fresh chats start warm.) Re-anchor long work. In a genuinely long session, an occasional one-paragraph recap — "where we are: X decided, Y open, draft three current" — plants the essentials at the end of the desk, the best-lit spot. Front-load what matters. Pasting a long document? Put your question and priorities before and after it, not buried mid-stack.

And a nice payoff: you can now read AI headlines like an insider. "Million-token context" means a bigger desk — more documents at once, longer sessions before drift — and says nothing about being smarter on any given sentence. Desk size and intelligence are different axes, whatever the launch post implies.

Try this in your chatbot: early in a chat, plant a detail — "my project codename is Falcon; remember it." Then chat about anything for a long stretch, paste something big, and eventually ask "what's my codename?" In one sitting it will likely hold; the point is watching how it holds — then asking it to summarise your whole conversation, and noticing what it kept and what it quietly dropped. That summary is the desk, shown to you.

← Previous