Agentic RAG

Agentic RAG gives the retrieval process to an AI agent — letting it decide when to retrieve, what to search for, how many times to retrieve, and whether the results are good enough before generating an answer.

---

Table of Contents

• What is Agentic RAG?
• How It Differs from Standard RAG
• Core Agentic RAG Patterns
• The Agentic RAG Loop
• Multi-Agent RAG
• Tools an Agentic RAG System Uses
• Challenges & Trade-offs
• Nano Banana Image Prompt
• Key Takeaway

---

What is Agentic RAG?

In standard RAG, retrieval is passive — it always fires once, retrieves top-K chunks, and hands them to the LLM. There is no judgment about whether the retrieved content is good, whether more retrieval is needed, or whether the question requires multi-step reasoning.

Agentic RAG makes retrieval active. An LLM agent controls the entire process:

• Should I retrieve at all, or do I already know the answer?
• What is the best search query for this question?
• Are the retrieved documents actually useful?
• Do I need to search again with a different query?
• Should I search a different source?

Standard RAG:   Query → Retrieve (once) → Generate → Done

Agentic RAG:    Query → Agent decides →
                         ├─ Retrieve? (maybe not needed)
                         ├─ Which tool? (vector DB, web, SQL)
                         ├─ Good enough? (self-critique)
                         ├─ Retrieve again? (refine query)
                         └─ Generate → Done

---

How It Differs from Standard RAG

Dimension	Standard RAG	Agentic RAG
Retrieval decision	Always retrieves	Agent decides if retrieval is needed
Number of retrievals	Fixed (1 round)	Dynamic (0, 1, or many rounds)
Query formulation	Direct from user input	Agent reformulates for best results
Result evaluation	None — uses whatever was retrieved	Agent critiques and rejects poor results
Tool selection	Single retriever	Chooses from multiple tools dynamically
Complexity	Low	High
Latency	Fast	Slower (multiple LLM calls)

---

Core Agentic RAG Patterns

Self-RAG

The model generates a reflection token to decide whether retrieval is needed, then critiques the retrieved documents and its own output.

Steps:
1. [Retrieve?]    → YES / NO  (model decides)
2. [Retrieve]     → Fetch documents
3. [Is Relevant?] → YES / NO  (model grades each document)
4. [Is Supported?]→ Model checks if its answer is grounded in docs
5. [Is Useful?]   → Model rates its own response quality
6. Retry or return final answer

Corrective RAG (CRAG)

Evaluates retrieved documents with a grader. If quality is too low, falls back to web search.

Retrieve from vector store
      │
      ▼
Grade documents (LLM)
      │
  ┌───┴───────────────────────────┐
  │                               │
Relevant ✅                  Not Relevant ❌
      │                               │
Generate answer            Web Search fallback
                                   │
                           Grade web results
                                   │
                           Generate answer

Multi-hop RAG

For complex questions that require chaining multiple retrievals:

Q: "What technique did the team that won the 2023 ML competition use for retrieval?"

Hop 1: Search → "Who won the 2023 ML competition?" → "Team Hydra won"
Hop 2: Search → "What retrieval technique did Team Hydra use?" → "Hybrid BM25 + dense"
Answer: "Team Hydra used hybrid BM25 + dense retrieval"

Adaptive RAG

Classifies the query complexity first, then routes to the appropriate retrieval strategy:

Query → Classifier
              ├─ [Simple]   → No retrieval (model already knows)
              ├─ [Single-hop] → Standard RAG
              └─ [Multi-hop]  → Agentic multi-step RAG

---

The Agentic RAG Loop

┌─────────────────────────────────────────────────────┐
│                  AGENTIC RAG LOOP                   │
│                                                     │
│   User Query                                        │
│       │                                             │
│       ▼                                             │
│   ┌──────────────────────────────────────────────┐  │
│   │              AGENT (LLM)                     │  │
│   │                                              │  │
│   │  1. Analyse query                            │  │
│   │  2. Plan retrieval strategy                  │  │
│   │  3. Call retrieval tool(s)                   │  │
│   │  4. Evaluate retrieved docs                  │  │
│   │  5. Refine query if needed → loop back       │  │
│   │  6. Generate final answer                    │  │
│   └──────────────────────────────────────────────┘  │
│                          │                          │
│          ┌───────────────┼───────────────┐          │
│          ▼               ▼               ▼          │
│     Vector DB       Web Search       SQL DB         │
└─────────────────────────────────────────────────────┘

---

Multi-Agent RAG

For very complex tasks, you can distribute RAG across specialised agents:

Orchestrator Agent
├── Research Agent     → Searches multiple sources, grades results
├── Synthesis Agent    → Combines retrieved info into coherent context
├── Generation Agent   → Produces the final answer
└── Critique Agent     → Validates the answer against sources

This is used in advanced applications like autonomous research assistants, due diligence tools, and complex Q&A systems.

---

Tools an Agentic RAG System Uses

Tool	Purpose
vector_search	Semantic search in private knowledge base
web_search	Live internet search fallback
sql_query	Query structured databases
document_reader	Parse and extract from raw files
grade_documents	LLM-based relevance scoring
rewrite_query	Reformulate query for better retrieval

---

Challenges & Trade-offs

Challenge	Detail
Latency	Multiple LLM calls per query → slower response
Cost	More token usage per query
Reliability	Agent can get stuck in loops if not managed
Debugging	Harder to trace why a specific retrieval decision was made
Mitigation	Set max iteration limits; add structured logging; use LangGraph for state management

---

Nano Banana Image Prompt

A macro shot of a tiny nano banana sitting in a control room with multiple screens, actively choosing which screen (labelled "Vector DB", "Web Search", "SQL") to retrieve from, while holding a checklist to grade the results. Clean white background, flat illustration style.

---

Key Takeaway

Agentic RAG puts an intelligent agent in control of the retrieval process — dynamically deciding when to retrieve, from where, and whether the results are good enough — unlocking the ability to answer complex, multi-step questions that standard RAG cannot handle.

---

← Modular RAG | RAG Overview