Ollama

The video is in Italian; this page is the written version in English.

Ollama became the de facto standard for local inference, and the reason fits on one line:

ollama run qwen3:8b

It downloads the model if missing, loads it, and drops you at a prompt. No signup, no account, no data leaving the machine.

It is a DVD player for models: instead of going to the cinema and buying a ticket every time, you take the film home and watch it as often as you like, with nobody knowing what you are watching.

A necessary clarification

Llama and Ollama look alike and are not the same thing. Llama is a model, made by Meta: the brain. Ollama is a runtime, made by a different company: the engine that runs models, including Qwen, DeepSeek, Mistral and dozens of others.

The commands you will actually use

ollama pull qwen3:8b        # download the model
ollama run qwen3:8b         # chat in the terminal
ollama list                 # what you have downloaded
ollama ps                   # what is loaded in memory right now
ollama rm qwen3:8b          # free up space

Names follow a model:size pattern, and with no further detail it pulls the default quantized variant — usually 4-bit, which is the right pick most of the time.

The part that matters: the API

Ollama exposes a local service on localhost:11434 that is OpenAI-API compatible. That single decision is what made it ubiquitous: any tool that speaks to OpenAI works with Ollama by changing the address.

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Explain photosynthesis in two lines",
  "stream": false
}'

From there you can build on it: scripts, applications, agents, editor extensions. All local.

Customising a model

A configuration file called a Modelfile creates a variant with your own system prompt and parameters, callable like any other model.

FROM qwen3:8b
PARAMETER temperature 0.3
SYSTEM "You are a Python code reviewer. Reply only with the problems you find."

It is the simplest way to turn a generic model into a specialised tool, with no training involved.

Things to know

The model stays loaded in memory for a few minutes after use, so follow-up requests start instantly; after the timeout it is unloaded. If you want different behaviour — always loaded, or never — an environment variable controls it.

Watch where the files go too: models weigh several gigabytes each and pile up quickly. It is worth checking now and then with ollama list.

Trap number one: the default context

The most common and hardest-to-diagnose problem: the default context window is far smaller than what the model supports.

A model advertising 32,000 tokens is served with a window of a few thousand unless you say otherwise. The rest raises no error: it is simply truncated.

Typical symptoms:
  - "forgets" the beginning of a long document
  - ignores instructions given a few messages earlier
  - answers about half the text you pasted
  - in long chats loses the thread always at the same point

The fix is raising the context parameter, in the model configuration file or in the call. But beware: a larger context costs memory, because the cache grows with it. If the system starts swapping to disk, you went too far.

Before blaming the model for amnesia, check how much context you are actually giving it.

Exposing it on the network: know this first

Ollama listens only on the local machine by default. Changing that to reach it from another device is convenient, and should be done knowing one thing: the API has no authentication. Whoever reaches it can use the models, pull them and delete them.

Scenario	Risk
Local only (default)	None
Open on the home network	Anyone on the network uses it
Open on the internet	Never do this

The right way to use it from outside is the one that applies to all personal applications: a private network between your own devices, making it reachable only from inside without exposing anything. The general principle: only what is public goes through the public tunnel; anything holding your data stays on the private network.

In short

Concept	In one line
What it is	The standard runtime for running models locally
Llama vs Ollama	The first is a model, the second the engine that runs it
API	OpenAI-compatible on localhost:11434: anything can plug in
Modelfile	Creates variants with your own system prompt and parameters
Watch out	Models take gigabytes: check occasionally
Default context	Smaller than advertised, and truncates silently
Network	The API has no authentication: never expose it to the internet