Advanced RAG

Advanced RAG improves on the basic retrieve-and-generate pipeline by adding smarter query handling, better retrieval, and post-retrieval processing — significantly improving accuracy and relevance.

---

Table of Contents

• What is Advanced RAG?
• Pre-Retrieval Techniques
• Retrieval Techniques
• Post-Retrieval Techniques
• When to Use Advanced RAG
• Nano Banana Image Prompt
• Key Takeaway

---

What is Advanced RAG?

Naive RAG follows a simple linear flow: embed query → retrieve → generate. It works but has weaknesses:

• Poor queries return irrelevant chunks
• Retrieved chunks may be noisy or redundant
• The LLM gets too much or too little context

Advanced RAG adds intelligent processing at three stages: before retrieval, during retrieval, and after retrieval.

Naive RAG:
  Query ──► Embed ──► Search ──► LLM ──► Answer

Advanced RAG:
  Query ──► [Pre-Retrieval] ──► [Retrieval] ──► [Post-Retrieval] ──► LLM ──► Answer
             (rewrite, expand)   (hybrid, HyDE)   (re-rank, compress)

---

Pre-Retrieval Techniques

Improve the query before it hits the vector store:

Query Rewriting

Rephrase the user's query to be more specific and retrieval-friendly.

User: "Why is my app slow?"
Rewritten: "What are common causes of high latency in web applications?"

Query Expansion

Generate multiple variations of the query to broaden retrieval coverage.

Original: "How to reduce LLM costs?"
Expanded:
  - "Techniques to lower OpenAI API spend"
  - "Optimising token usage in language models"
  - "Caching strategies for LLM applications"

HyDE (Hypothetical Document Embeddings)

Ask the LLM to generate a hypothetical ideal answer first, embed that, then use it to search — finding documents that look like good answers rather than good questions.

Query: "What is the capital of Australia?"
Hypothetical answer: "The capital of Australia is Canberra, located in the ACT..."
→ Embed the hypothetical answer → Search → Retrieve real documents that match

---

Retrieval Techniques

Hybrid Search

Combine dense (semantic) search and sparse (keyword/BM25) search for better coverage:

Dense search:   Catches semantically similar content
Sparse search:  Catches exact keyword matches

Combined score = α × dense_score + (1-α) × sparse_score

Best for: domains with specific terminology (medical, legal, technical).

Contextual Chunking

Add document-level context to each chunk before embedding, so the chunk knows where it came from:

Original chunk:
  "The process takes 3–5 business days."

Contextual chunk:
  "From: Acme Corp Refund Policy (Section 4 — Processing Times)
   The process takes 3–5 business days."

Parent-Child Chunking

Store small chunks for precise retrieval, but return their parent (larger) chunk to the LLM for richer context.

Index:     [small chunk 1] [small chunk 2] [small chunk 3]
                    ↑                ↑
Retrieve:  small chunk → fetch parent → send full section to LLM

---

Post-Retrieval Techniques

Re-ranking

Use a cross-encoder to re-score the top-K retrieved chunks for true relevance before sending to the LLM. More accurate than vector similarity alone.

Vector search returns: [chunk A, chunk B, chunk C, chunk D, chunk E]  (top-5)
Cross-encoder re-ranks: [chunk C, chunk A, chunk E]  (top-3 most relevant)
→ Send only top-3 to LLM

Contextual Compression

Extract only the relevant sentences from each retrieved chunk, reducing noise in the prompt.

Retrieved chunk (500 tokens):  "... lots of unrelated text ... 
                                The refund window is 7 days. 
                                ... more unrelated text ..."

After compression (12 tokens): "The refund window is 7 days."

Lost in the Middle Mitigation

LLMs attend better to content at the beginning and end of the context window. Place the most relevant chunks first and last, not in the middle.

---

When to Use Advanced RAG

Situation	Recommended Technique
Users ask vague or ambiguous questions	Query rewriting / expansion
Domain has lots of jargon/acronyms	Hybrid search (BM25 + dense)
Retrieved chunks often miss the point	HyDE
Too much noise in retrieved context	Re-ranking + compression
Long documents, need precise + rich context	Parent-child chunking

---

Nano Banana Image Prompt

A macro shot of a tiny nano banana wearing a detective hat, holding a magnifying glass over a stack of glowing documents, carefully selecting the most relevant one to hand to a large mechanical brain. Clean white background, flat illustration style.

---

Key Takeaway

Advanced RAG dramatically improves retrieval quality by adding intelligent query handling before retrieval, smarter search during retrieval, and noise reduction after retrieval — closing the gap between what users ask and what the LLM actually needs.

---

← RAG Overview | Modular RAG →