Transformers · Intuition series · Lab 3

One geometry, six jobs

Semantic search, recommendations, classification, clustering, deduplication, anomaly detection. They're taught as six techniques. They are one computation — who is near whom — with a different question asked of it. Here are all six, running on the same seventeen points.

Lesson progress 0%

The whole idea, in one table

Once your data is vectors, you have a distance table: how close is everything to everything else. Every product below reads that one table. What changes is the question — and, in code, usually a single line.

Job	The question it asks of the distance table
Search	Who is nearest this query? Rank them.
Recommend	Who is nearest this item? Rank them. (Identical call. Different noun.)
Classify	What label do my nearest neighbours carry? Take a vote.
Cluster	What groups exist when there is no query and no labels at all?
Dedup	Which pairs are too near? Above a threshold, call them the same thing.
Anomaly	Who is near nothing? Below a threshold, flag it.

Why this framing is worth having People learn these as six separate tools with six separate libraries. They aren't. If you can compute similarity and sort, you can build all six — and RAG is just row one plus "paste the results into a prompt." The hard part of a retrieval system was never the retrieval; it's whether the geometry puts the right things near each other.

The workbench

The same 17 words from the explainer, in 3-D. Switch jobs and watch what changes — the overlay and the ranking change; the underlying numbers never do. Drag to rotate; click a word to select it.

Query

Neighbours (k) · 3

Clusters (k) · 4

Threshold · 0.60

what just ran

Similarity metric

Bonus job: arithmetic on meaning

Because meaning is stored as directions, you can do algebra with it: isolate a direction by subtracting, then add it somewhere else and ask who's nearest. Same distance table, one extra step.

−+=?

Honest note: this lands perfectly here because the 17 points were hand-placed so the gender direction is exact — king − man + woman is literally queen's coordinates. On real embeddings analogies work for a few relation types (gender, capital-of, plurals) and fail for most others, and the standard benchmark quietly excludes the input words from the answer, which does a lot of the work. Real, but narrower than the famous demos suggest.

Where this toy lies to you

Everything above is honest about mechanism and dishonest about difficulty. Three things are far too easy here:

The groups are clean. Every word's nearest neighbour is in its own group — 17 out of 17. Real corpora have overlapping, ragged, unbalanced groups.
Classification is trivially perfect — until it isn't. k-NN scores 17/17 leave-one-out at k=1, 3 and 5. Push the slider to k=7 and it collapses to 9/17. Nothing broke: three of the four groups hold only four words, so at k=7 a point's own group cannot supply a majority of its seven neighbours. k must stay below your smallest class size — a real constraint this toy happens to make visible.
Three dimensions is not two hundred. Distances behave intuitively here. In high dimensions they concentrate — everything drifts toward equidistant — which is why nearest-neighbour search needs specialised indexes.

But one thing here is completely real Switch the metric toggle in the workbench and watch the dedup threshold break. Under centered cosine there's a clean window (0.435 → 0.800) where "same group" and "different group" never overlap. Under raw cosine that window is gone — within-group similarity dips to 0.958 while between-group climbs to 0.973. Same points, same groups, different metric, and your threshold silently starts merging unrelated things. Choosing the metric is not a formality.

So what is RAG, exactly?

Row one, twice, plus string concatenation:

chunks = split(documents) vectors = embed(chunks) # build the distance table, once — q = embed(user_question) # the query becomes a point hits = top_k(vectors, q, k=5) # ← this is "Search". That's the R in RAG. answer = llm("Context:\n" + join(hits) + "\n\nQuestion: " + user_question)

Every retrieval bug you will ever hit lives in one line — the top_k. If the geometry puts the wrong things near the query, no amount of prompt engineering downstream will save the answer. Which is why it's worth being able to look at the space.

Check yourself

Six jobs, one table. These questions are about the parts that are easy to nod along to and hard to actually hold — including the two places this page's own demo will contradict you if you guess.

Five questions across the six jobs.

The lesson at a glance

✓ Grounded in published work

Where these ideas come from

The seventeen points on this page are hand-placed and illustrative. The six algorithms running on them are the real ones, and the claims about how they behave at scale come from these:

Efficient Estimation of Word Representations in Vector Space — Mikolov et al., 2013 · word2vec; the "king − man + woman" result the bonus section demos
Issues in Evaluating Semantic Spaces Using Word Analogies — Linzen, 2016 · why analogies are narrower than the famous demos suggest, and what excluding the input words does
When Is "Nearest Neighbor" Meaningful? — Beyer, Goldstein, Ramakrishnan & Shaft, 1999 · distance concentration in high dimensions
Efficient and Robust Approximate Nearest Neighbor Search Using HNSW Graphs — Malkov & Yashunin, 2016 · the index a real search system uses instead of scanning
Billion-scale Similarity Search with GPUs — Johnson, Douze & Jégou, 2017 · FAISS; the same six jobs at production scale
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020 · the paper that named the last section

Six products, one distance table.

Where this sits: the embeddings explainer is how a word becomes a vector; the embedding lab is that geometry on real word2vec vectors. This page is what you build once you have it.

This is one of the interactive explainers we make at AgenticGHX. If it was useful, the rest — talks, learning tracks, and research out of Ghana — is one tap away.