What Large Language Models Actually Are

An LLM is a program that has “read” billions of pages of text and learned to predict the next word. It does not understand, reason or think: it recognises statistical patterns in language with startling accuracy. Everything else — the useful answers, the working code, the translations — follows from that single ability.

The video is in Italian; this page is the written version in English.

The core idea: predicting the next token

Read this sentence: “The cat sat on the ___”. What goes there? Probably “sofa”, “table”, “mat”. Your brain has seen thousands of similar sentences and knows which words are likely.

An LLM does exactly this, at industrial scale. It has read hundreds of billions of words — books, websites, code, conversations — and for every position in a sentence it has learned the probability distribution of what comes next. When it generates, it draws the next token from that distribution, then uses it to draw the one after, and so on.

Input:  "The capital of France is"

Output: " Paris"        probability ~95%
        " a city"       probability ~2%
        " located"      probability ~1%
        ...             thousands of other options with tiny probabilities

The model does not “know” that Paris is the capital of France. It has seen that association so many times in training data that the statistical probability is overwhelming. A subtle difference, but a decisive one — and it explains most of the mistakes.

A short timeline

Year	Model	Parameters	What changed
2017	Transformer	—	The architecture everything since is built on
2018	GPT-1	117M	Generic pre-training, then task-specific fine-tuning
2019	GPT-2	1.5B	Text so good it was not released straight away
2020	GPT-3	175B	Few-shot: examples in the prompt, no retraining
2023	GPT-4	~1.8T	Multimodal, stronger reasoning
2023	Llama 2	7–70B	Open weights: the local AI ecosystem is born
2024	Llama 3	8–405B	Quality close to closed models, openly available
2025	DeepSeek R1	671B MoE	Step-by-step reasoning with open weights
2025	Qwen 3	0.6–235B	Thinking and non-thinking modes in one model

Two moments explain most of it. In 2017 the paper Attention Is All You Need introduced the Transformer — the transistor moment for this field. In 2023 Meta released Llama with downloadable weights, and suddenly a capable model could run on a laptop instead of in a data centre.

Generative or discriminative?

This distinction clears up a lot of confusion.

A discriminative model classifies: give it an email and it says “spam” or “not spam”, give it a photo and it says “cat”. The output is a label, picked from a closed set.

A generative model produces new content: text, images, audio. The output is not picked from a list, it is built piece by piece.

LLMs are generative. When an assistant replies “Yes, this is a spam email”, it is not classifying: it is generating the sentence “Yes, this is a spam email”, one token at a time. Classification is a side effect of generation — which is why the reasoning that comes with it is sometimes invented on the spot.

Base, Instruct, Chat: why the suffix matters

When you download a model you will often see these variants, and the difference is practical.

The base model is the raw one, straight out of pre-training. It can predict the next token but cannot hold a conversation: ask “What is the capital of France?” and it may happily continue with “What is the capital of Germany? What is the capital of Spain?”, because it has seen plenty of question lists. It is a starting point for further training, not for daily use.

The instruct model is the base plus training to follow instructions. It answers questions, performs tasks, stays on point. Ideal for one-shot operations: summarise, translate, fix.

The chat model adds optimisation for dialogue: it keeps track across turns, adapts its tone, handles back-and-forth.

Same prompt — “Explain photosynthesis simply” — three behaviours: the base carries on like a textbook without addressing you, the instruct gives a correct dry definition, the chat reaches for an analogy and offers to go deeper.

What 7B, 70B and MoE mean

Parameters are the numbers that make up the model. More parameters means more capability, but also more memory to run it.

Parameters	Class	Rough memory (4-bit quantised)
0.5–3B	Nano	1–2 GB
7–8B	Small	4–5 GB
13–14B	Medium	8–9 GB
32–34B	Large	18–20 GB
70B	XL	35–40 GB
405B+	XXL	200+ GB

On a laptop with 16 GB of memory, an 8-billion-parameter model quantised to 4 bits is the sweet spot: around 5 GB and quick to answer. Move up to 32B and you start swapping to disk, which makes the conversation painful.

Then there is an architectural trick that changes the arithmetic: Mixture of Experts. An MoE model has a very large number of total parameters but activates only a fraction of them per token.

Dense 70B model    → uses all 70B parameters for every token
MoE 141B model     → has 141B in total, activates ~40B per token
                     quality near a 70B, speed near a 40B

That is why models that look enormous on paper can run on far more modest hardware than the headline number suggests.

The part that matters most: the model does not understand

If you take one thing away, make it this. An LLM is an extraordinarily sophisticated parrot: it has heard billions of conversations and reproduces linguistic patterns with impressive fidelity, but it does not “understand” the way we do.

When an assistant writes “I’m sorry, I can’t help with that”, it feels no regret: it has seen thousands of similar replies and is reproducing the pattern that fits.

There is a simple test. Ask what 2+2 is and the answer is 4 — correct, but not because the model can add. Ask what 2847593 + 3948271 is and it often gets it wrong, because it has not seen that particular combination enough times. Models that reason explicitly, writing out the steps before answering, do much better; the underlying mechanism is still statistical token prediction.

Four practical consequences follow:

1. They can make things up (hallucinations): they generate the most likely continuation, not the truest one.
2. They are excellent with languages, because that is literally the job.
3. They improve with context: the more useful information in the prompt, the better the answer.
4. They have no memory between conversations: everything they “remember” lives in the context window.

Three words that are not synonyms

Much of the confusion around these tools comes from calling four different things “artificial intelligence”. Separating them is worth the effort, because knowing which layer you are talking about tells you where to look when something misbehaves.

Thing	What it really is	Example
Model	The weights and architecture. Alone it does nothing, it must be run	Qwen3, Llama
Chatbot	The interface on top: it manages history and formatting	ChatGPT, Open WebUI
Agent	A loop that uses the model to decide actions and then performs them	assistants that write files
Generative AI	The broader set, images and audio included	image generators

The most expensive confusion is between model and chatbot. When people say “the AI doesn’t remember me”, it is not the model that forgot: it is the interface that stopped resending the conversation. The model, for its part, never remembered anything.

When the model thinks before answering

The timeline above stops at models that answer in one go. Since 2025 there is a separate category: those that generate intermediate reasoning before the final answer.

CLASSIC MODEL
  question → [one pass] → answer

REASONING MODEL
  question → [reasoning, sometimes thousands of tokens] → answer

The underlying mechanism does not change: it is still next-token prediction. What changes is that the model uses its own tokens as scratch paper, exactly as we work sums out on paper instead of in our heads. Each written step becomes context for the next.

In practice: much better on multi-stage problems — maths, logic, code, multiple constraints — and much slower and more expensive, because reasoning consumes tokens, often more than the answer itself. On simple tasks it is just added waiting.

One warning worth keeping: displayed reasoning is not a guarantee. It can be impeccable and lead to a wrong conclusion, or rationalise an answer picked beforehand. A powerful aid, not a proof.

Why it fails at arithmetic and letter counting

Two limits that look absurd in a system that writes working code, and that share one cause: the model does not see letters or digits the way we do.

"How many r's are in strawberry?"

What the model sees:  ["straw", "berry"]
What would be needed: ["s","t","r","a","w","b","e","r","r","y"]

Counting letters means opening up chunks that are atomic to the model. The same goes for long numbers, split in unpredictable ways, leaving digits misaligned for addition.

The countermeasure is simple and general: when symbolic precision matters, the model should write the tool rather than do the calculation. Ask for the procedure, or better the code, and run that. It is also why assistants able to execute programs are far more reliable on numerical tasks: they do not compute, they delegate.

What actually changes the quality you experience

With the same model, experience varies enormously because of factors that have nothing to do with the model. In order of real impact:

Factor	Impact
How the prompt is written	very high
How much relevant context you supply	very high
Model variant (base, instruct, chat)	high
Generation parameters	medium-high
Quantization level	medium
Parameter count	medium

The counterintuitive part is the last row compared with the first: moving from an 8-billion model to a 14-billion one matters less than writing a better prompt. People complaining about local model quality are almost always using prompts designed for frontier models.

In short

Concept	In one line
LLM	A model that predicts the next token from statistical patterns
Generative	Creates new content instead of assigning a label
Base	Can predict, cannot converse
Instruct	Trained to follow instructions
Chat	Optimised for multi-turn dialogue
Parameters	The numbers that make up the model (7B = 7 billion)
MoE	Activates only part of the parameters per token
Hallucination	False but plausible output, generated because it is likely
Model and chatbot	The first is the weights, the second the interface: do not conflate them
Explicit reasoning	Tokens become scratch paper: better on complex problems
Symbolic precision	Have the model write code rather than do the maths