WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/embedding-space/

1
00:00:00.067 --> 00:00:10.815
To compare words, a model turns each one into a point, placed so that nearness means similar meaning: here are forty-two of them, in three dimensions.

2
00:00:10.815 --> 00:00:21.918
Train that space well and the points gather by meaning: the animals sit together, the professions sit together, and the capitals hover just above their countries.

3
00:00:21.918 --> 00:00:34.012
Trace one word: ask for the nearest neighbours of king, and uncle, prince and father come back, each with a similarity above 0.98.

4
00:00:34.012 --> 00:00:52.402
The surprise is that directions carry meaning: man to woman is the same arrow as king to queen, so king minus man plus woman lands exactly on queen, and every country to its capital is one more parallel arrow.

5
00:00:52.402 --> 00:01:12.187
Similarity is the angle between arrows from the origin, not the gap between the points: king and queen are four units apart and forty-four degrees apart, a similarity of 0.72, while prince, at ten degrees, scores 0.98.

6
00:01:12.187 --> 00:01:27.966
The catch: real embeddings have hundreds or thousands of dimensions, and every picture drops most of them, so flatten just one here and each capital lands on its country, and each queen on her king.

7
00:01:27.966 --> 00:01:39.988
So rank neighbours by cosine, in the full space, and use a picture like this only for the shape of the groups, never for how close two words really are.