How AI Models Understand You: Tokens, Context and Hallucinations

Once you can write a clear prompt, the next level is understanding what happens to it inside the model — not the mathematics, but the concepts and the English that describes them. These are the terms that come up in every serious conversation about AI, and using them precisely marks the difference between someone who has read a headline and someone who has thought about the tool.

Tokens

A model does not read words. It reads tokens — chunks of text, roughly a word or part of a word. unhappy might be one token; unbelievable might be two.

• The model splits the text into tokens.
• This model has a large token limit.

The useful consequence is token limit — the most the model can take in or produce at once. When a prompt is "too long", what has run out is tokens.

Context window

The context window is how much the model can hold in mind at once — its short-term memory, measured in tokens. Everything in your conversation, plus the prompt, sits inside it.

• The whole document fits inside the context window.
• Once the conversation exceeds the context window, the model forgets the beginning.

This is why long conversations with a chatbot sometimes lose the thread: the beginning has fallen out of the window. In careful English you say the model runs out of context, not that it forgot — forgetting implies memory, which a model does not have between sessions.

Temperature

Temperature controls how predictable the output is. It borrows the idea from physics — higher temperature means more movement.

• Low temperature — predictable, safe, literal. Good for facts and code.

• High temperature — varied, surprising, creative. Good for ideas and drafts.

• Set the temperature low for a technical answer.

• The model's temperature was high, so it gave three very different drafts.

The word is now standard enough that people say the response felt high-temperature — meaning unpredictable — without explanation.

Hallucination

A hallucination is a confident-sounding answer that is simply false. The model invented a fact, a name, a source, and stated it as if it were certain.

• The model hallucinated a citation.
• That reference is a hallucination — it doesn't exist.

This is the term every careful user of AI needs, because it names the tool's central failure. The verb is regular: hallucinate, hallucinated, has hallucinated. And the noun is count: a hallucination, two hallucinations.

Hallucination is not lying. Lying requires knowing the truth and choosing against it. A model does not know what is true; it predicts what text would follow. Calling a false output a hallucination is precise in a way lie and mistake are not — the model was never checking reality to begin with.

The rest of the set

These complete the core, and you meet them alongside precise technical vocabulary:

Term	Meaning
training data	the texts the model learned from
parameters	the numbers the model learned; "size" in technical terms
inference	the moment the model produces an answer
prompt injection	an attack hidden inside a prompt that makes the model ignore its instructions
fine-tuning	adjusting a model for one specific job

Talking about it precisely

The pattern to notice: each term has a verb that goes with it, and the pairing is fixed the way collocations always are.

• The model splits text into tokens.
• The answer exceeded the context window.
• I lowered the temperature.
• The model hallucinated a source.

Using the right verb signals that you understand the mechanism, not just the words.

Test yourself

The quiz below checks the terms and the verbs that go with them.