What is an embedding?
Computers do not speak English or Dutch; they speak in numbers. To make a language model like GPT or Claude understand a word, we must translate that word into a mathematical form. We call this process an embedding.
An embedding is essentially a long list of numbers (a vector) that captures the meaning of a word, sentence, or entire document. Instead of seeing text as random letters, the AI calculates where this text belongs in a massive virtual dictionary of meanings.
The vector space: An intuitive explanation
Imagine a large 3D space. In this space, we place words based on their properties.
- Words like apple, banana, and pear float very close to each other, in the 'fruit corner'.
- Words like dog, cat, and rabbit form their own cluster further away, in the 'animal corner'.
When an AI 'reads', it translates your text into coordinates in this space. The closer two coordinates are to each other, the more closely related they are in meaning. In reality, this space does not have 3 dimensions, but often 1,536 to more than 4,000 dimensions to capture all the nuances of human language (such as tone, context, and grammar).
Cosine Similarity: Measuring distance
To determine how similar two texts are, we measure the angle between their vectors. This is called Cosine Similarity. A score of 1 means identical meaning, a score around 0 means no relationship at all, and -1 points to the exact opposite.
Key applications
Because embeddings capture meaning, they form the backbone of modern AI applications:
- Semantic Search (Vector Search): Instead of searching for exact keywords, you search by concept. If you search for "how to boil an egg", the system will also find texts containing "preparing eggs in boiling water". This is essential for Retrieval-Augmented Generation (RAG) systems.
- Clustering: Automatically grouping thousands of customer service tickets based on the underlying issue, without the texts needing to contain the same words.
- Recommendations: Suggesting articles or products that conceptually lie in the same "space" as the items a user has previously viewed.
Exercise: Think in distances
Look at the following three sentences. Reason which two would lie closest to each other in the vector space:
- "The bank was closed due to a system outage."
- "I went fishing by the river bank yesterday."
- "Due to an IT issue, we couldn't withdraw money."
Answer: Sentences 1 and 3 lie closest to each other. Although sentences 1 and 2 share the word "bank", the embedding model understands the context. Sentences 1 and 3 are both about financial institutions and technical issues, causing their semantic vectors to overlap.
Check out the technical documentation on api.llmnet.nl and learn how to generate your own embeddings. →