Background
Back to Articles/vLLM & LLM Inference Serving
Sections
IntroductionOrientation01 · Why Inference Servers Exist02 · Where vLLM Sits03 · Anatomy of a vLLM SetupStage 1 — Core Concepts01 · The KV Cache02 · Prefill vs Decode03 · PagedAttention04 · Continuous Batching05 · Sampling ParametersStage 2 — The Engine01 · The Scheduler & Block Manager02 · Lifecycle of a Request03 · Prefix Caching04 · Quantisation05 · Speculative DecodingStage 3 — Running It Locally01 · Installing vLLM02 · Offline Batch Inference03 · The OpenAI-Compatible Server04 · Streaming & Client Patterns05 · The Getting-Started NotebookStage 7 — Case Studies01 · RAG Chatbot at Scale02 · Offline Batch Summarisation03 · Coding Agent on Long Context04 · Multi-Tenant LoRA on One GPU

04 · Multi-Tenant LoRA on One GPU

3 min read

Status: 📋 Stub. Title and brief only; content not yet written.

Shape: cost-constrained · many small tenants · isolation matters

A SaaS product that offers every customer a fine-tuned model. Thirty customers today, each with a LoRA adapter over the same base model, each generating a modest and wildly uneven amount of traffic — two customers account for most of it, and eleven send fewer than a hundred requests a day. The budget does not stretch to a GPU per customer, and the honest reason a GPU per customer is wrong isn't cost alone: twenty-eight of those GPUs would sit idle.

This is the study where the interesting questions stop being about tokens per second and start being about tenancy. One noisy customer can consume the whole block pool. A shared server means a shared blast radius for a bad deploy. And there is a real crossover point — a number of tenants, or a traffic concentration — past which separate deployments genuinely become the cheaper and safer answer, which this page works out rather than asserts.

Exercises: multi-LoRA serving from one base model · adapter loading and hot-swap · where LoRA memory actually goes · per-tenant rate limiting and fair queuing · auth and isolation at the gateway · cost per million tokens per tenant · the shared-versus-dedicated crossover.

Runnable artefact: scripts/04-tenancy-cost-model.py — no GPU required. Takes a tenant traffic distribution and derives cost per tenant under shared and dedicated topologies, and finds the crossover point for your own numbers.

The Brief

📋

Architecture

📋

First Attempt

📋

What Broke

📋

The Diagnosis

📋

The Fix

📋

The Numbers

📋 All figures on this page are modelled, not measured — see the stage note.

What Changes at 10×

📋

Lessons

📋

Cheat Sheet

📋

← Previous: Coding agent on long context · Back to Case Studies · Glossary & Cheat Sheet →

⚠️ Verification checklist (delete before publishing)

  • Page is a stub. Nothing below the brief has been written or verified.
  • Verify the current multi-LoRA support surface against the docs — max adapters, rank limits, whether --enable-lora flags have changed, and whether hot-swap without restart is supported on the target version.
  • vLLM has no built-in per-tenant fairness; confirm this before the page recommends solving it at the gateway.
  • The shared-versus-dedicated crossover must fall out of the script, not be asserted as a round number.