Compression
Retrieved chunks contain irrelevant sentences. Passing 5 full chunks (2500 tokens) to the LLM wastes context window, increases cost, and adds noise that causes hallucination. Compression extracts query-relevant excerpts — often reducing context by 60-80% with no quality loss.
Compression is like highlighting only the relevant paragraphs in five textbook chapters before an exam — you keep the signal, ditch the filler.
Visual Workflows
Start here — scroll inside each diagram frame to explore, then use + / − to zoom up to 200% if needed.
Overview
Scroll inside the frame to explore · use + / − to zoom up to 200%
Scroll inside the frame to explore · use + / − to zoom up to 200%
Retrieve then squeeze
Scroll inside the frame to explore · use + / − to zoom up to 200%
Broad retrieval, then keep only query-relevant spans.
Token budget
Scroll inside the frame to explore · use + / − to zoom up to 200%
Compression is a context-engineering tool: spend tokens on evidence, not filler.
Key Takeaways
- 1.Context compression reduces retrieved chunks to only the sentences relevant to the query before passing them to the LLM — fitting more information into limited context windows while reducing noise and cost.
- 2.Techniques: (1) LLMLingua / LongLLMLingua — prompt compression via token pruning.
- 3.(2) Extractive compression — embed sentences, keep top-k most similar to query.
- 4.(3) Abstractive — LLM summarizes each chunk relative to query.
- 5.(4) Contextual compression retriever (LangChain) — wraps retriever with compression step.
Real Example
Scenario
Five 500-token policy chunks retrieved. Compression extracts 15 most query-relevant sentences (400 tokens total). LLM receives focused context, generates accurate answer at 40% lower input token cost.
What you would do
In RAG Engineering, apply Compression to this scenario: Five 500-token policy chunks retrieved. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.
Practice Task
Open the Code Walkthrough below and run it locally. Change one parameter related to Compression (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.
Code Walkthrough
Highlighted lines show where Compression happens in the code.
1from langchain.retrievers.document_compressors import EmbeddingsFilter # import dependencies2from langchain.retrievers import ContextualCompressionRetriever # import dependencies3
4compressor = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=0.76)5compression_retriever = ContextualCompressionRetriever( # key line for Compression6 base_compressor=compressor,7 base_retriever=vectorstore.as_retriever(search_kwargs={"k": 10}),8)9
10compressed_docs = compression_retriever.invoke("What is the refund window?") # key line for CompressionCommands
Commands to Remember
ContextualCompressionRetriever(base_retriever, compressor)Compress after retrieval, not instead of retrievalEval faithfulness — aggressive compression drops citationsSavings show up in prompt tokens, not in the vector DB
Cheat Sheet
Quick recap
quick ref- •Compress after re-rank
- •Extractive = safe for facts
- •similarity_threshold=0.76 start
- •Measure token reduction + quality
- •Skip for small chunks
- •ContextualCompressionRetriever
Common Mistakes
- ✕Abstractive compression altering factual content
- ✕Compressing before re-ranking — wastes compute
- ✕Over-compressing — removing critical context
- ✕Not measuring faithfulness after compression
- ✕Breaking citation mapping with abstractive summaries