RAG

LLMs have knowledge cutoffs, hallucinate facts, and can't access private data. RAG solves this by retrieving relevant information at query time and injecting it into the prompt — giving the LLM access to up-to-date, domain-specific, and proprietary knowledge.

RAG is like an open-book exam — instead of relying on memory (training data), the student (LLM) looks up relevant pages (retrieved documents) and writes an answer based on what they find.

Visual Workflows

Start here — study each diagram, then use + / − to zoom if needed.

100%

Loading diagram...

Scroll inside the frame · use + / − to zoom

Key Takeaways

1.RAG (Retrieval-Augmented Generation) combines information retrieval with LLM generation — fetching relevant documents and using them as context to produce grounded, accurate answers.
2.RAG pipeline: (1) Ingestion — load documents, chunk, embed, store in vector DB.
3.(2) Retrieval — embed user query, find top-k similar chunks.
4.(3) Generation — inject chunks into prompt, LLM generates answer grounded in context.
5.Variants: naive RAG (basic retrieve + generate), advanced RAG (query rewriting, re-ranking, hybrid search), modular RAG (separate retrieval and generation optimization).

Real Example

Scenario

An employee asks 'What's our parental leave policy?' The system retrieves the relevant HR document chunks, includes them in the prompt, and GPT-4 generates an accurate answer citing the specific policy section.

What you would do

In RAG Engineering, apply RAG to this scenario: An employee asks 'What's our parental leave policy?' The system retrieves the relevant HR document chunks, includes them in the prompt, and GPT-4 generates an accurate answer citing the specific policy section. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Practice Task

Open the Code Walkthrough below and run it locally. Change one parameter related to RAG (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Code Walkthrough

Highlighted lines show where RAG happens in the code.

RAG

1from openai import OpenAI  # import dependencies2import chromadb  # import dependencies3
4client = OpenAI()  # create API client5chroma = chromadb.Client()6collection = chroma.get_or_create_collection("hr_docs")7
8def rag_query(question: str) -> str:  # define a reusable function9    results = collection.query(query_texts=[question], n_results=3)10    context = "\n\n".join(results["documents"][0])  # key line for RAG11
12    response = client.chat.completions.create(  # call the API13        model="gpt-4o-mini",14        messages=[15            {"role": "system", "content": "Answer based only on the provided context."},  # key line for RAG16            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},  # key line for RAG17        ],18        temperature=0,19    )20    return response.choices[0].message.content  # return the result

Cheat Sheet

Quick recap

quick ref

•Chunk → Embed → Store → Retrieve → Generate
•ChromaDB/Pinecone for storage
•Top-k=3-5 typical
•temp=0 for factual answers
•Evaluate retrieval + generation
•Ground answers in context

Common Mistakes

✕Skipping retrieval evaluation — only testing final answers
✕Chunks too large (dilute relevance) or too small (lose context)
✕Not instructing LLM to use only provided context
✕Ignoring retrieval failures and blaming the LLM