Pub/Sub

In a Nutshell

Publish/Subscribe (pub/sub) is a messaging pattern where publishers send messages to a topic rather than to specific recipients, and any number of subscribers interested in that topic each receive their own copy. The publisher doesn't know or care who's listening — it just announces "this happened," and everyone subscribed hears it. This one-to-many, fully-decoupled model is the backbone of event-driven architectures: a single event (an order was placed, a user signed up) can trigger many independent reactions across the system without the publisher being wired to any of them.

How It Actually Works

Queue (Point-to-Point) vs Pub/Sub (Broadcast)

This is the fundamental distinction from a plain message queue:

	Message Queue (P2P)	Pub/Sub
Delivery	Each message → one consumer	Each message → every subscriber
Relationship	Competing consumers share work	Independent subscribers each react
Analogy	A task list workers pull from	A newsletter everyone receives
Use	Distribute work	Distribute events/notifications

Queue:    msg → [ worker A | worker B | worker C ]   only ONE picks it up
Pub/Sub:  event → topic → A gets it, B gets it, C gets it  (all of them)

The Core Components

• Publisher — emits messages to a topic; knows nothing about subscribers.
• Topic (or channel/subject) — a named stream of messages; the point of indirection that decouples both sides.
• Subscriber — registers interest in a topic and receives every message published to it.
• Broker — the infrastructure (Kafka, Redis, SNS, Google Pub/Sub) that routes messages from topics to subscribers.

The topic is the key: because publishers and subscribers only know the topic, either side can change, scale, or fail independently. You can add a new subscriber to react to an existing event without touching the publisher at all.

Push vs Pull Delivery

Model	How	Example
Push	Broker delivers messages to subscribers as they arrive	SNS → HTTP/Lambda, WebSockets
Pull	Subscribers poll/fetch messages at their own pace	Kafka consumers, SQS

Pull gives subscribers control over throughput (backpressure); push gives lower latency but can overwhelm slow subscribers.

Filtering and Fan-Out

Subscribers often don't want every message on a topic. Two refinements:

• Topic-based — subscribe to a whole topic (orders).
• Content/attribute-based — subscribe with a filter (orders where region = EU), so the broker only delivers matching messages.

Fan-out is the defining superpower: one published event triggers many parallel downstream flows.

"order.created" published once →
   ├─ email service      → send confirmation
   ├─ inventory service  → decrement stock
   ├─ analytics pipeline → record event
   ├─ fraud service      → risk-check
   └─ recommendation svc → update model
Add a 6th reaction later? Just subscribe — publisher unchanged.

Durability and Delivery Semantics

Pub/sub systems vary in how they treat messages:

System	Retention Model
Redis Pub/Sub	Fire-and-forget — offline subscribers miss messages
Kafka	Durable log — messages retained; subscribers replay from any offset
SNS/SQS, Google Pub/Sub	Durable with per-subscriber delivery + retries + DLQ

This matters: Redis Pub/Sub is ephemeral (great for live fan-out like a WebSocket backplane, bad if a subscriber must never miss an event), whereas Kafka's retained log lets subscribers go offline and catch up later, or replay history entirely.

Seeing It in Action

Scenario: Event-driven user onboarding via pub/sub.

# --- Publisher: the signup service ---
def register_user(email, name):
    user = db.create_user(email, name)
    # Publish ONE event. The signup service knows nothing about who reacts.
    broker.publish("user.signed_up", {
        "user_id": user.id, "email": email, "name": name,
        "region": user.region, "ts": now(),
    })
    return user            # returns immediately; reactions happen async

# --- Independent subscribers, each owning one concern ---
@subscribe("user.signed_up")
def send_welcome_email(event):    email_service.welcome(event["email"])

@subscribe("user.signed_up")
def sync_to_crm(event):           crm.upsert(event)

@subscribe("user.signed_up")
def start_free_trial(event):      billing.begin_trial(event["user_id"])

# Content-filtered subscriber — only EU users
@subscribe("user.signed_up", filter="region == 'EU'")
def gdpr_consent_flow(event):     compliance.request_consent(event["user_id"])

Why this design scales organizationally, not just technically: each team owns a subscriber for its own concern (email, CRM, billing, compliance) and can deploy, scale, and fail independently. The signup service stays simple — it emits one event and is done. Adding a new onboarding step (say, a Slack notification to the sales team) means adding a subscriber, with zero changes to the publisher or the other subscribers. This loose coupling is why pub/sub underpins microservice and event-driven architectures.

Interview Questions

1. Q: How does pub/sub differ from a point-to-point message queue? Hint: In a queue, each message is consumed by exactly one consumer (competing consumers share work). In pub/sub, each message is delivered to every subscriber (one-to-many broadcast). Queues distribute work; pub/sub distributes events. The publisher in pub/sub doesn't know who's subscribed — it publishes to a topic, and the broker fans out to all subscribers.

2. Q: Why is the "topic" the key to pub/sub's decoupling? Hint: Publishers and subscribers only reference the topic, never each other. This indirection lets either side change, scale, or fail independently, and lets you add new subscribers to react to existing events without modifying the publisher. It's what makes pub/sub the backbone of extensible event-driven architectures.

3. Q: What is fan-out, and give an example of its value. Hint: Fan-out is one published event triggering many independent downstream reactions in parallel. E.g., "order.created" simultaneously drives email confirmation, inventory decrement, analytics, fraud checks, and recommendations — each owned by a different subscriber/team. New reactions are added by subscribing, with no change to the publisher, enabling independent development and scaling.

4. Q: Compare push and pull delivery in pub/sub. Hint: Push: the broker delivers messages to subscribers as they arrive — low latency, but can overwhelm slow subscribers. Pull: subscribers fetch at their own pace — gives them backpressure/throughput control (Kafka, SQS), at some latency cost. Choose pull when consumers vary in speed or need flow control; push for low-latency notification-style delivery.

5. Q: How do Redis Pub/Sub and Kafka differ in durability, and when does it matter? Hint: Redis Pub/Sub is fire-and-forget — offline subscribers miss messages (great for ephemeral live fan-out like WebSocket backplanes). Kafka is a durable retained log — subscribers track offsets, can go offline and catch up, or replay history. It matters when a subscriber must never miss an event or needs to reprocess: use a durable log (Kafka, SNS/SQS, Google Pub/Sub), not ephemeral pub/sub.

References

• Google Cloud Pub/Sub overview — topics, subscriptions, delivery
• AWS SNS documentation — pub/sub with fan-out to SQS/Lambda
• Enterprise Integration Patterns — the canonical pattern

Dive Deeper

• Kafka: The Definitive Guide — durable pub/sub via a distributed log
• Martin Fowler: What do you mean by "Event-Driven"? — event patterns and pitfalls
• Google Pub/Sub architecture — a globally-scaled pub/sub system