0
0Document Loaders1Chunking2Embedding Models3Vector Databases4Metadata5Retrievers6Hybrid Search7BM258Cross Encoder9Re-ranking10Query Expansion11Compression12Caching13RAG14LangChain Basics15ChromaDB16Streamlit17RAG Evaluation
RAG Engineering
Phase 3Module 12 of 18

Compression

Retrieved chunks contain irrelevant sentences. Passing 5 full chunks (2500 tokens) to the LLM wastes context window, increases cost, and adds noise that causes hallucination. Compression extracts query-relevant excerpts — often reducing context by 60-80% with no quality loss.

Compression is like highlighting only the relevant paragraphs in five textbook chapters before an exam — you keep the signal, ditch the filler.

Visual Workflows

Start here — study each diagram, then use + / − to zoom if needed.

100%
Loading diagram...

Scroll inside the frame · use + / − to zoom

Key Takeaways

  • 1.Context compression reduces retrieved chunks to only the sentences relevant to the query before passing them to the LLM — fitting more information into limited context windows while reducing noise and cost.
  • 2.Techniques: (1) LLMLingua / LongLLMLingua — prompt compression via token pruning.
  • 3.(2) Extractive compression — embed sentences, keep top-k most similar to query.
  • 4.(3) Abstractive — LLM summarizes each chunk relative to query.
  • 5.(4) Contextual compression retriever (LangChain) — wraps retriever with compression step.

Real Example

Scenario

Five 500-token policy chunks retrieved. Compression extracts 15 most query-relevant sentences (400 tokens total). LLM receives focused context, generates accurate answer at 40% lower input token cost.

What you would do

In RAG Engineering, apply Compression to this scenario: Five 500-token policy chunks retrieved. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Practice Task

Open the Code Walkthrough below and run it locally. Change one parameter related to Compression (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Code Walkthrough

Highlighted lines show where Compression happens in the code.

Compression
1from langchain.retrievers.document_compressors import EmbeddingsFilter  # import dependencies2from langchain.retrievers import ContextualCompressionRetriever  # import dependencies3
4compressor = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=0.76)5compression_retriever = ContextualCompressionRetriever(  # key line for Compression6    base_compressor=compressor,7    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 10}),8)9
10compressed_docs = compression_retriever.invoke("What is the refund window?")  # key line for Compression

Cheat Sheet

Quick recap

quick ref
  • •Compress after re-rank
  • •Extractive = safe for facts
  • •similarity_threshold=0.76 start
  • •Measure token reduction + quality
  • •Skip for small chunks
  • •ContextualCompressionRetriever

Common Mistakes

  • ✕Abstractive compression altering factual content
  • ✕Compressing before re-ranking — wastes compute
  • ✕Over-compressing — removing critical context
  • ✕Not measuring faithfulness after compression
  • ✕Breaking citation mapping with abstractive summaries