Stage 7 — Case Studies

You can take a workload description and produce a defensible architecture, config and cost model — end to end. · ~1 week

Every earlier stage taught one thing at a time. Real systems don't arrive one thing at a time. They arrive as a sentence from someone who doesn't know what a KV cache is — "we need an internal assistant over our docs, a few hundred people will use it, and it has to feel fast" — and your job is to turn that into a model choice, a GPU, a set of flags, a deployment, an alert and a number in pounds per million tokens.

This stage is four complete projects. Not vignettes: each page is a whole buildable system, with an architecture diagram, the actual vLLM configuration, the surrounding code, the deployment manifests, the numbers, and — the part most write-ups omit — the first attempt that didn't work and the log line that told you why.

The four studies

They're chosen to sit at four corners of the design space, so that between them they exercise nearly every knob in the article. Each one makes a different trade-off look obvious, and the instincts you build in one will be actively wrong in another. That's deliberate: the fastest way to learn that "latency matters" is not a universal truth is to read Study 1 and then Study 2.

#	Project	Shape	Status
1	RAG chatbot at scale	Latency-bound, high concurrency, heavy prefix reuse	📋
2	Offline batch summarisation	Throughput-only, latency irrelevant, cost-dominated	📋
3	Coding agent on long context	Long context, bursty, KV-cache-pressure-bound	📋
4	Multi-tenant LoRA on one GPU	Cost-constrained, many small tenants, isolation matters	📋

#	The project in one paragraph
1	An internal RAG assistant for a 2,000-person company. ~200 concurrent users at peak, a 4k-token system prompt shared by every request, retrieved context appended per query, and a product requirement that the first token appears in under 800 ms. Exercises prefix caching, chunked prefill, max_num_seqs, and autoscaling on queue depth rather than CPU.
2	Two million support tickets summarised overnight into a warehouse table. Nobody is waiting; the only metrics are total wall-clock time and cost. Exercises the offline LLM class, quantisation, max_num_batched_tokens, spot instances and Airflow orchestration. Every latency instinct from Study 1 is wrong here, and the page says so explicitly.
3	An agentic coding assistant working over a repository: 60k-token contexts, most of it identical between consecutive turns, arriving in bursts when a developer is mid-task and then silent for twenty minutes. Exercises long context, chunked prefill, preemption, prefix cache hit rate as the single metric that decides everything, and speculative decoding.
4	A SaaS product serving thirty customer-specific fine-tunes of one base model on a single GPU, on a budget that doesn't allow a GPU per customer. Exercises multi-LoRA serving, per-tenant rate limiting and isolation, cost-per-token arithmetic, and the crossover point where separate pods become cheaper than shared.

The shape of a case-study page

These pages do not use the standard nine-section template. That template is built around learning one concept through an experiment; these pages are built around a system arriving broken and being reasoned into shape. The skeleton is:

Section	The question it answers
The Brief	What was actually asked for: workload, traffic shape, SLOs, budget, constraints
Architecture	The whole system as a diagram, then a walk through each component and why it's there
First Attempt	The configuration a reasonable engineer writes first — honestly, not a strawman
What Broke	The symptom, the log line, the metric. Written as you'd actually encounter it
The Diagnosis	The reasoning from symptom to cause, linked back to the page that teaches it
The Fix	The config diff and the code, with each change justified separately
The Numbers	Before and after: throughput, TTFT, p99, KV cache utilisation, cost per million tokens
What Changes at 10×	Which decision breaks first, and what the next architecture looks like
Lessons	The transferable conclusions, each linked to its source page
Cheat Sheet	The config block worth stealing, and the two numbers worth remembering

About the numbers

Be clear about what these figures are. Throughput, TTFT and utilisation numbers in this stage are modelled, not measured — derived from published memory-bandwidth and FLOPs figures, model config.json values, and the capacity arithmetic taught in The KV Cache and Memory & Capacity Tuning. They are labelled as modelled wherever they appear, and the derivation is a script you can run.

That's a weaker claim than a benchmark and a stronger one than a blog post's unlabelled table. The point of the numbers here is not that your A10G will produce exactly these figures; it's that you can see which term dominates, and substitute your own hardware and traffic and get your own answer.

Each study ships a no-GPU Python script under scripts/ that reproduces its capacity and cost model. Change the model, the GPU, the concurrency or the token distribution at the top of the file and the whole study re-derives.

Where to read this stage

It's placed last because every study leans on every earlier stage. But if you learn better from a finished system than from first principles, read Study 1 first, accept that half of it is unexplained, and come back after Stage 0. The lessons list at the bottom of each page is a reading list into the rest of the article.

---

← Back to Ecosystem & Comparisons · Glossary & Cheat Sheet →

⚠️ Verification checklist (delete before publishing)

• All four case-study pages are stubs — titles and briefs only. No content verified.
• Confirm the case-study page template is reflected in CLAUDE.md and the landing page's page-shape section before writing content.
• Every number in this stage must carry a "modelled, not measured" label and a link to the script that derives it. Nothing here has been derived yet.
• Relative links to 06-ecosystem/README.md and glossary-and-cheatsheet.md are currently dead — those pages don't exist yet.