Embeddings and vector space

The video is in Italian; this page is the written version in English.

Picture a library where books are ordered not by author or title but by meaning: a novel about loneliness sits next to a poem about melancholy, even though they share no words. The embedding is the librarian deciding where each book goes based on what it says, not how it is written.

Or think of GPS coordinates. Every place has a latitude and a longitude, and nearby places have similar numbers. Embeddings do the same for text.

What it is, concretely

An embedding model takes text and returns a vector: a fixed-length list of numbers.

"King"   → [ 0.23, -0.45,  0.82,  0.15, -0.67, ...]
"Queen"  → [ 0.25, -0.42,  0.79,  0.18, -0.63, ...]   similar numbers
"Car"    → [-0.56,  0.78, -0.12,  0.91,  0.23, ...]   distant numbers

Each number is a dimension of meaning. Nobody assigns them by hand: the model learns them during training. You can imagine them as “how regal is this concept”, “how concrete versus abstract”, “how positive” — multiplied by hundreds or thousands of axes.

Model	Dimensions
Word2Vec (2013)	300
nomic-embed-text	768
BGE-M3 (multilingual)	1024
OpenAI text-embedding-3-small	1536
OpenAI text-embedding-3-large	3072

Closeness means similarity

If meaning becomes a position, similarity becomes a distance. The usual measure is cosine similarity: it looks at the angle between two vectors, not their length. It is 1 when they point the same way, 0 when they are perpendicular.

Hence the most striking property: in embedding space you can do arithmetic on concepts.

vector("King") - vector("man") + vector("woman")  ≈  vector("Queen")

Not magic, geometry: the direction separating “man” from “woman” is roughly the one separating “king” from “queen”, because those pairs appear in analogous contexts.

What it is actually for

Semantic search. Search “how do I cut my household spending” and a keyword engine demands those words; embedding search also finds a document titled “saving on utility bills”. No shared words, same meaning.

RAG. This is the basis of retrieval augmented generation: split documents into chunks, embed each one, store them in a vector database. When a question arrives, find the nearest chunks and put them in the model’s context. That is how an LLM answers about documents it never saw in training.

Clustering and deduplication. Similar articles cluster in the same region of space, so you can group them or spot duplicates without reading them.

Things to know before using them

The embedding model must be the same for documents and queries: different models produce incompatible spaces and the comparison is meaningless. Change model and you have to recompute the whole archive.

Chunk size matters too: too large dilutes meaning, too small fragments it. A few hundred words with some overlap is the usual starting point.

When similarity is not enough

Meaning-based search has one precise weakness: words that must match exactly.

Query: "error E-4021 on the payment module"

Semantic search:  finds documents about payment errors in general,
                  but may miss the one containing exactly "E-4021"
Keyword search:   finds E-4021 immediately, but does not grasp
                  that "failed transaction" is the same topic

Codes, acronyms, proper nouns, version numbers, legal references: cases where literal matching beats meaning. The answer is not to choose but to combine. The two result lists merge into a single ranking, and in practice hybrid beats pure vector search almost every time: it costs little and recovers exactly the cases where vectors fail embarrassingly.

Two passes instead of one

There is a structural limit in this mechanism: question and document become vectors separately, never seeing each other. The comparison happens between two numerical summaries, not between the texts.

A reranker does the opposite: it takes the question-document pair together and scores relevance. Far more accurate and far slower, so it cannot run over a whole archive. Hence the two-stage pattern, standard in serious systems:

1. RETRIEVAL  fast       → from 100,000 documents take 50 candidates
2. RERANK     accurate   → among the 50, pick the 5 that truly fit
3. ANSWER     those 5 in the model's context

The gain is double: fewer chunks in context, and more relevant ones. Since context is the scarce resource, going from ten mediocre documents to three good ones improves the answer and cuts the cost at once.

The decision that outweighs the model

Before computing any vector you must decide how to split documents, and that choice matters more than which embedding model you use. It is almost always made without thinking.

Strategy	When
Fixed length	Homogeneous text, simple to do
By structure (headings, paragraphs)	Documentation, manuals, notes
By meaning	Discursive text, costs more

Two rules solve most cases. Keep an overlap between consecutive chunks, so a sentence straddling the boundary is not decapitated. And keep the source context with the chunk — title, section, date: an isolated paragraph says far less than one that knows where it came from.

The typical flaw is the opposite of what people expect: chunks that are too large. A whole chapter produces a vector averaging too many topics, so it sits close to no question in particular.

In short

Concept	In one line
Embedding	The meaning of a text turned into a list of numbers
Vector space	A place where closeness is similarity of meaning
Cosine similarity	The standard measure: the angle between two vectors
RAG	Search by meaning and put the retrieved chunks in the context
Rule of thumb	Same model for indexing and querying, always
Hybrid search	Keyword plus semantic: recovers the codes vectors miss
Reranking	An accurate second pass over few candidates
Chunking	The choice that outweighs the embedding model