Embeddings

Computers can't compare meaning of sentences directly. Embeddings convert text to vectors where semantically similar content is close in vector space. This powers RAG retrieval, recommendation, deduplication, and classification.

Embeddings are like GPS coordinates for meaning — 'king' and 'queen' are close on the map, 'king' and 'banana' are far apart. You find similar concepts by measuring distance between coordinates.

Visual Workflows

Start here — study each diagram, then use + / − to zoom if needed.

100%

Loading diagram...

Scroll inside the frame · use + / − to zoom

Key Takeaways

1.Embeddings are dense numerical vectors that capture semantic meaning of text — enabling similarity search, clustering, and retrieval in AI applications.
2.Embedding models (text-embedding-3-small, Cohere embed, open-source models) are neural networks trained to map text to fixed-size vectors (384-3072 dimensions).
3.Similar meanings → similar vectors.
4.Distance metrics: cosine similarity (most common), Euclidean distance, dot product.
5.Embeddings are deterministic for the same input.

Real Example

Scenario

You embed 10,000 support articles. A user asks 'How do I reset my password?' — the query embedding is closest to articles about password recovery, even though the exact words differ.

What you would do

In Generative AI Foundations, apply Embeddings to this scenario: You embed 10,000 support articles. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Practice Task

Open the Code Walkthrough below and run it locally. Change one parameter related to Embeddings (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Code Walkthrough

Highlighted lines show where Embeddings happens in the code.

Embeddings

1from openai import OpenAI  # import dependencies2import numpy as np  # import dependencies3
4client = OpenAI()  # create API client5
6def embed(text: str) -> list[float]:  # define a reusable function7    response = client.embeddings.create(  # call embedding model — text becomes a vector8        model="text-embedding-3-small",  # key line for Embeddings9        input=text,10    )11    return response.data[0].embedding  # the numeric vector representing meaning12
13def cosine_similarity(a: list[float], b: list[float]) -> float:  # measure how similar two vectors are (0 to 1)14    a, b = np.array(a), np.array(b)15    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))  # return the result16
17doc = embed("How to reset your password")18query = embed("I forgot my login credentials")19print(f"Similarity: {cosine_similarity(doc, query):.3f}")  # show output for debugging

Cheat Sheet

Quick recap

quick ref

•client.embeddings.create()
•cosine_similarity(a, b)
•text-embedding-3-small
•Similar meaning = close vectors
•Batch embed for efficiency
•Store in vector DB

Common Mistakes

✕Using different embedding models for indexing vs querying
✕Not normalizing vectors before cosine similarity
✕Embedding entire documents instead of chunks for RAG
✕Ignoring embedding model version changes on re-index