How LLMs Work

7 min read

Tokens, next-word prediction, and context windows — the three ideas that explain every chat AI you have ever used.

The models behind chat assistants have exactly one superpower, and it sounds almost insultingly simple: guess the next word-piece. That is the whole trick. Do it well enough, across trillions of examples, and the side effects include translation, poetry, working code, and — most relevant for you — dramatically better prompts for your images and videos. Let us unpack the three ideas that make it work.

Tokens: the LEGO bricks of language

An LLM never sees letters or words. It sees tokens — chunks of text that average about 4 characters, or roughly three-quarters of an English word. "Generative" might split into gener + ative; a common word like "the" is a single token. Each token maps to a number, because neural networks only eat numbers. Fun fact for Hebrew speakers: non-English languages usually cost *more* tokens per sentence, since tokenizers were trained mostly on English text.

One job: guess the next token

Training works like the world's largest fill-in-the-blank exam. The model — a transformer network — reads mountains of text, repeatedly predicts the next token, and every time it guesses wrong, its billions of parameters get nudged toward the right answer. That humble objective quietly forces the model to absorb grammar, facts, styles, and logic, because you cannot reliably finish the sentence "the capital of France is…" without actually knowing things.

• Translation — because it saw the same ideas expressed in many languages.
• Summarizing and rewriting — because it learned what "the same thing, shorter" looks like.
• Writing code — because GitHub is text too.
• Polishing your image prompts — because it read millions of great descriptions.

Context window: the model's working memory

The context window is how much text the model can consider at once, measured in tokens. In 2026, mainstream models handle hundreds of thousands of tokens — entire books in one conversation. But two catches: anything outside the window is invisible to the model, and the window is *working* memory, not permanent memory — a fresh chat starts from zero. And because the model predicts likely text rather than looking up verified facts, it can confidently invent things. That failure mode is called a hallucination, and it stars in Capabilities and Limits.

One more dial worth knowing: temperature. Low temperature makes the model pick the safest, most likely next token every time — predictable and repetitive. Higher temperature lets it take chances — more creative, occasionally weirder. Same model, different personality.

Rewrite this rough idea as a detailed image generation prompt with a clear subject, setting, lighting, and style: a cozy coffee shop on a rainy evening

Related glossary terms: LLM (Large Language Model), Token, Context Window, Transformer, Temperature

Let an LLM sharpen your ideas into professional prompts. Open the Prompt Builder