Phase 3Module 14 of 22

RAG

LLMs have knowledge cutoffs, hallucinate facts, and can't access private data. RAG solves this by retrieving relevant information at query time and injecting it into the prompt — giving the LLM access to up-to-date, domain-specific, and proprietary knowledge.

RAG is like an open-book exam — instead of relying on memory (training data), the student (LLM) looks up relevant pages (retrieved documents) and writes an answer based on what they find.

Visual Workflows

Start here — scroll inside each diagram frame to explore, then use + / − to zoom up to 200% if needed.

Overview

100%

Loading diagram...

Scroll inside the frame to explore · use + / − to zoom up to 200%

100%

Loading diagram...

Scroll inside the frame to explore · use + / − to zoom up to 200%

End-to-end RAG

100%

Loading diagram...

Scroll inside the frame to explore · use + / − to zoom up to 200%

Private knowledge enters at retrieve time. The LLM should answer from context, not from memory.

What RAG fixes

100%

Loading diagram...

Scroll inside the frame to explore · use + / − to zoom up to 200%

Cutoff, private data, and citations. It does not fix bad chunks or a weak retriever.

Key Takeaways

1.RAG (Retrieval-Augmented Generation) combines information retrieval with LLM generation — fetching relevant documents and using them as context to produce grounded, accurate answers.
2.RAG pipeline: (1) Ingestion — load documents, chunk, embed, store in vector DB.
3.(2) Retrieval — embed user query, find top-k similar chunks.
4.(3) Generation — inject chunks into prompt, LLM generates answer grounded in context.
5.Variants: naive RAG (basic retrieve + generate), advanced RAG (query rewriting, re-ranking, hybrid search), modular RAG (separate retrieval and generation optimization).

Real Example

Scenario

An employee asks 'What's our parental leave policy?' The system retrieves the relevant HR document chunks, includes them in the prompt, and GPT-4 generates an accurate answer citing the specific policy section.

What you would do

In RAG Engineering, apply RAG to this scenario: An employee asks 'What's our parental leave policy?' The system retrieves the relevant HR document chunks, includes them in the prompt, and GPT-4 generates an accurate answer citing the specific policy section. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Practice Task

Open the Code Walkthrough below and run it locally. Change one parameter related to RAG (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Code Walkthrough

Highlighted lines show where RAG happens in the code.

RAG

1from openai import OpenAI  # import dependencies2import chromadb  # import dependencies3
4client = OpenAI()  # create API client5chroma = chromadb.Client()6collection = chroma.get_or_create_collection("hr_docs")7
8def rag_query(question: str) -> str:  # define a reusable function9    results = collection.query(query_texts=[question], n_results=3)10    context = "\n\n".join(results["documents"][0])  # key line for RAG11
12    response = client.chat.completions.create(  # call the API13        model="gpt-4o-mini",14        messages=[15            {"role": "system", "content": "Answer based only on the provided context."},  # key line for RAG16            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},  # key line for RAG17        ],18        temperature=0,19    )20    return response.choices[0].message.content  # return the result

Commands

Commands to Remember

RAG = retrieve relevant chunks, then generate from that context
System: Answer only using the provided context. Cite sources.
temperature=0 for factual RAG
If retrieval is wrong, generation cannot be faithful

Cheat Sheet

Quick recap

quick ref

•Chunk → Embed → Store → Retrieve → Generate
•ChromaDB/Pinecone for storage
•Top-k=3-5 typical
•temp=0 for factual answers
•Evaluate retrieval + generation
•Ground answers in context

Common Mistakes

✕Skipping retrieval evaluation — only testing final answers
✕Chunks too large (dilute relevance) or too small (lose context)
✕Not instructing LLM to use only provided context
✕Ignoring retrieval failures and blaming the LLM