Similarity Search

RAG depends on finding the right context for each question. Similarity search matches query embeddings to document embeddings, returning the most relevant chunks. Poor search = wrong context = bad LLM answers.

Similarity search is like asking a librarian 'find me books similar to this one' — they don't match titles word-for-word but find books on the same topic by comparing content themes.

Visual Workflows

Start here — study each diagram, then use + / − to zoom if needed.

100%

Loading diagram...

Scroll inside the frame · use + / − to zoom

Key Takeaways

1.Similarity search finds the most semantically related items in a vector database by comparing embedding distances — the core retrieval mechanism in RAG systems.
2.Process: embed query → compute distance to all (or indexed subset of) stored vectors → return top-k nearest neighbors.
3.Distance metrics: cosine similarity (1 = identical, 0 = orthogonal), Euclidean distance (lower = closer), dot product.
4.Parameters: k (how many results), score threshold (minimum similarity), metadata filters.
5.Challenges: semantic drift (query and document use different words), multi-hop reasoning (answer spans multiple chunks), and recency bias (old but relevant docs ranked lower without metadata filtering).

Real Example

Scenario

User asks 'What's the warranty on electronics?' Similarity search returns chunks about 'product guarantee for devices' and 'electronic item coverage period' — semantically matching despite different wording.

What you would do

In Generative AI Foundations, apply Similarity Search to this scenario: User asks 'What's the warranty on electronics?' Similarity search returns chunks about 'product guarantee for devices' and 'electronic item coverage period' — semantically matching despite different wording. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Practice Task

Open the Code Walkthrough below and run it locally. Change one parameter related to Similarity Search (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Code Walkthrough

Highlighted lines show where Similarity Search happens in the code.

Similarity Search

1import numpy as np  # import dependencies2
3def search(query_vec, doc_vectors, doc_texts, top_k=3):  # define a reusable function4    scores = []5    for i, doc_vec in enumerate(doc_vectors):6        sim = np.dot(query_vec, doc_vec) / (  # key line for Similarity Search7            np.linalg.norm(query_vec) * np.linalg.norm(doc_vec)  # key line for Similarity Search8        )9        scores.append((sim, doc_texts[i]))10    scores.sort(reverse=True)11    return scores[:top_k]  # return the result12
13# Usage with pre-computed embeddings14results = search(query_embedding, doc_embeddings, doc_texts, top_k=5)  # key line for Similarity Search15for score, text in results:16    print(f"[{score:.3f}] {text[:80]}...")  # show output for debugging

Cheat Sheet

Quick recap

quick ref

•cosine_similarity(query, doc)
•top_k=5 typical for RAG
•score threshold filters noise
•precision@k for evaluation
•Bi-encoder = fast retrieval
•Cross-encoder = accurate reranking

Common Mistakes

✕Setting k too high — flooding LLM with irrelevant context
✕No score threshold — returning low-quality matches
✕Not evaluating retrieval quality separately from generation
✕Using wrong distance metric for your embedding model