Cloud or local?

The video is in Italian; this page is the written version in English.

It is like choosing where to cook. At home: you buy the equipment once and then cook whenever you like without paying anyone, but you have to buy it. At a restaurant: you pay only for the dishes you order, no kitchen to maintain, but you do not choose the ingredients. A professional kitchen by the hour: better equipment than yours, and you pay while you use it.

Local

You install a runtime, download the model, and it runs on your machine. The upfront cost is hardware — which you may already own — and the recurring cost is just electricity.

Privacy is total: no data leaves the device, which changes everything when confidential documents, customer data or medical material are involved. It also works with no network.

The limits are clear: open-weights models only, and speed depends on what you have. An 8B on recent hardware answers at a comfortable pace; going bigger means going bigger on hardware too.

Through an API

You pay as you go, usually per token, and reach the most capable closed models. No maintenance, instant scalability, and top-tier quality that local models reach some time later.

In exchange: data leaves your machine, you depend on someone else’s availability and pricing, and a model you use in production can be retired or change behaviour under your feet. At high volume, the bill grows fast.

Rented GPUs

The middle road: rent a machine with powerful cards and run what you like on it, including large open-weights models. It makes sense for training, batch jobs, or intense but time-limited experiments. You pay by the hour, so shut it down when idle.

How to actually decide

If…	Go with
The data is confidential	local
You need the best quality available	API
Volume is high and steady	local, if the hardware holds
Volume is low or spiky	API
You need to train or fine-tune	rented GPUs
You want to experiment and learn	local

The economics work like this: estimate monthly token consumption, multiply by API pricing, compare with hardware cost spread over two or three years plus electricity. Below a certain threshold the API always wins; above it, local pays for itself.

The sensible answer: both

In practice the combination beats an exclusive choice almost every time. Local for daily work, sensitive data, experimentation and anything not requiring the very top. API for the hard tasks where quality genuinely matters.

The hidden benefit of this setup is that it forces you to work out which tasks really need the best model. Usually far fewer than you would think.

The cascade: small first, large only if needed

The hybrid approach gets far more effective when the choice is made not once but per request.

request
   │
   ▼
small local model  ── solved? ──► done (zero cost)
   │ no
   ▼
large model via API  ──► answer

On real workloads most requests are simple: rewrites, classifications, extractions, questions about documents you already supplied. If the local model handles even half of them, spending halves with nobody noticing the difference.

The delicate part is deciding when to escalate: by task type, set in advance with a rule; by confidence declared by the small model, which flags when it is out of its depth; or by output verification, if an automatic check can say whether the answer is acceptable.

Cost to account for: two systems to maintain and keep aligned. It makes sense above a certain scale, not for personal use.

What happens when a model disappears

An under-considered risk of building on APIs: models get retired. They ship, age, and at some point stop being served, with notice that rarely matches your timeline. And even without retirement, an update can change behaviour under the same name.

Three proportionate precautions: pin the version when the provider allows it, rather than pointing at the alias that follows the latest release; keep your own test cases, the only way to notice something changed before your users do; and isolate the call behind a function of your own, so swapping provider or model is a change in one place.

It is also one of the concrete arguments for local: a downloaded model does not shift under your feet and does not expire. What runs today will run identically in two years.

In short

Mode	Cost	Privacy	When
Local	hardware + power	total	sensitive data, steady volume, learning
API	pay per use	data leaves	top quality, low or spiky volume
Rented GPU	hourly	medium	training, batch jobs
Cascade	The small model handles most requests, the large one the rest
Continuity	APIs change and retire models; a downloaded file does not