Modular RAG

Modular RAG breaks the retrieval pipeline into independent, swappable components — giving you full control to mix, replace, or extend each stage without rebuilding the whole system.

---

Table of Contents

• What is Modular RAG?
• The Module Library
• Common Modular Patterns
• Modular vs. Naive vs. Advanced RAG
• When to Use Modular RAG
• Nano Banana Image Prompt
• Key Takeaway

---

What is Modular RAG?

In Naive and Advanced RAG, the pipeline is mostly linear and tightly coupled. Modular RAG takes a different approach — it treats each stage of the pipeline as an independent, interchangeable module.

You can:

• Swap one retriever for another without touching the rest
• Add a new module (e.g., a memory module) between any two stages
• Run modules in parallel, sequence, or conditionally
• Compose entirely custom pipelines for your specific use case

Naive RAG:    [Query] → [Retriever] → [Generator]   (fixed, linear)

Modular RAG:  [Query] → [Search Module]
                               ↓
                      [Filter Module]
                               ↓
                      [Re-rank Module]
                               ↓
                      [Memory Module]
                               ↓
                      [Generator Module]   (composable, swappable)

---

The Module Library

Modular RAG defines a catalogue of reusable modules:

Search Modules

Handle how content is retrieved from the knowledge base.

Module	Description
Dense Retriever	Semantic search using vector embeddings
Sparse Retriever	Keyword search using BM25/TF-IDF
Hybrid Retriever	Combines dense + sparse
Web Search	Live internet retrieval (Tavily, SerpAPI)
SQL Retriever	Queries structured databases
Graph Retriever	Traverses knowledge graphs

Memory Modules

Manage what the system remembers across turns.

Module	Description
Conversation Memory	Recent chat history in context window
Entity Memory	Tracks key entities mentioned by user
Summary Memory	Compresses old history into summaries
Long-term Memory	External vector store for persistent recall

Processing Modules

Transform, filter, or enrich retrieved content.

Module	Description
Query Rewriter	Improves the query before retrieval
Re-ranker	Scores retrieved chunks by relevance
Compressor	Extracts only relevant sentences
Fusion	Merges results from multiple retrievers

Generation Modules

Control how the LLM produces its output.

Module	Description
Standard Generator	Basic LLM call with context
Chain-of-Thought	Forces step-by-step reasoning
Citation Generator	Appends source references to output
Self-Critique	Model reviews and revises its own answer

---

Common Modular Patterns

Pattern 1: Sequential (Standard)

Modules execute one after another in a fixed order.

Query → Rewriter → Hybrid Retriever → Re-ranker → Compressor → LLM → Answer

Pattern 2: Conditional (Routing)

Route the query to different retrieval paths based on its type.

Query → Classifier
              ├─ [Factual]     → SQL Retriever → LLM
              ├─ [Semantic]    → Dense Retriever → Re-ranker → LLM
              └─ [Current]     → Web Search → LLM

Pattern 3: Parallel (Fusion)

Run multiple retrievers simultaneously, then merge results.

Query ──→ Dense Retriever ──→ ┐
       ──→ Sparse Retriever ──→ Fusion Module → Re-ranker → LLM
       ──→ Web Search ────────→ ┘

Pattern 4: Iterative (Multi-hop)

Loop through retrieval multiple times, using each result to refine the next query.

Query → Retrieve → Partial Answer → New Query → Retrieve → Final Answer

---

Modular vs. Naive vs. Advanced RAG

Dimension	Naive RAG	Advanced RAG	Modular RAG
Pipeline structure	Linear, fixed	Linear + enhancements	Composable, graph-like
Flexibility	Low	Medium	High
Complexity	Low	Medium	High
Custom routing	❌	❌	✅
Parallel retrieval	❌	❌	✅
Swappable components	❌	⚠️ Partial	✅
Best for	Prototyping	Production quality	Complex enterprise systems

---

When to Use Modular RAG

✅ You need to handle multiple query types differently (factual vs. conversational vs. real-time)
✅ You want to A/B test different retrievers or re-rankers
✅ Your use case requires parallel retrieval from multiple sources
✅ You need enterprise-grade flexibility without rewriting the whole pipeline
✅ You're building a platform where different teams plug in their own modules

---

Nano Banana Image Prompt

A macro shot of a tiny nano banana standing at a LEGO-like assembly table, snapping together colourful modular pipeline blocks labelled "Retriever", "Re-ranker", "Memory", and "Generator" into a custom chain. Clean white background, flat illustration style.

---

Key Takeaway

Modular RAG gives you maximum flexibility by treating every step of the RAG pipeline as a swappable, composable module — enabling conditional routing, parallel retrieval, and enterprise-grade customisation without coupling.

---

← Advanced RAG | RAG Overview | Agentic RAG →