05 · The Getting-Started Notebook

Seventeen pages of theory, arithmetic and flags. This page ships the artifact that runs all of it: one Colab notebook that installs vLLM, generates offline, serves online, streams, and measures — in the order the article taught it.

→ vllm-getting-started.ipynb

It has a second job, which is why it's more useful than a typical quickstart. Every cell captures a number this article currently states as unverified, and the final cell prints them all in one block to paste back. Running it once both proves your setup works and closes most of the verification run sheet.

Sections dropped on this page. The template's Idea, Under the Hood, Dial It In, In Production and Check Yourself sections aren't here. This is an artifact, not a concept: there's no analogy to build, no mechanism hidden underneath, no knobs of its own, and the notebook itself is the self-check. Padding those sections would be worse than dropping them.

---

The Problem

• You've read the theory and never seen it run. Every number so far has been arithmetic or someone else's measurement.
• The article has holes it admits to. Output blocks marked # UNVERIFIED, claims flagged as reasoned rather than measured, and three pages asserting a cancellation behaviour nobody confirmed.
• The experiments are scattered across seventeen pages, each needing its own server flags, in an order that isn't obvious from any one page.
• Colab has its own failure modes — a torch that gets replaced under you, sessions that disconnect, one GPU that can't hold two engines at once — none of which are vLLM's fault and all of which will waste your afternoon.

---

What's in it

Ten sections, each mapped to the page it verifies:

#	Cell	Verifies	Page
0	Hardware check	Compute capability ≥ 7.5, disk, Python	Installing vLLM
1	Capacity arithmetic	The predicted concurrency figure (247 @ 4096)	Anatomy
2	Install	Version pinning; the Colab restart	Installing vLLM
3	Startup log	Predicted vs actual GPU blocks — the overhead term, measured	PagedAttention
4	Chat template	That generate() vs chat() differs visibly	Offline Batch
5	Batching	Batched-vs-looped ratio; the n = 1…64 sweep	Why Inference Servers Exist
6	KV formula	Whether predicted/measured really is 1.000	The KV Cache
7	Serve + render	The /v1/chat/completions/render payload shape	The Server
8	Streaming	TTFT rises with prompt length; whether ITL stays flat	Prefill vs Decode
9	Preemption	Whether raising max_num_seqs under pressure makes it worse	The Scheduler
10	Results	Prints everything as JSON to paste back	—

The three in bold are the ones worth prioritising if you only have one session — each settles a claim the article currently makes on reasoning alone, and each would require a real edit if it came back the other way.

Cell 3 is the one to read carefully

It closes the loop opened in Stage 0. You predicted 247 concurrent sequences from published metadata on a laptop; the engine reports its real block count at startup. The difference between them is the overhead term — activations, CUDA graphs, framework — that capacity.py estimated at 1 GB.

Whatever that gap turns out to be, it's the honest error bar on every capacity figure in this article, and it's currently unquantified.

---

Try It

Runtime → Change runtime type → T4 GPU, then run top to bottom.

Two things that will interrupt you, both expected:

Cell 2 will prompt for a restart. Take it. Colab ships its own torch and vLLM replaces it — that's the printer-driver problem, not a bug. After restarting, skip cells 0–2 and continue from cell 3.

Cell 3 asks you to paste a number back. The GPU-blocks value from the startup log goes into BLOCKS, then re-run that cell to get the predicted-versus-actual comparison. It's manual because the log's exact wording varies by version, and grepping for it would break silently.

If you're short on time

Cells 0, 1, 3, 6 and 9 in that order. That's the capacity prediction, the real block count, the KV formula check, and the preemption experiment — the four measurements the article most needs.

What to send back

Cell 10 prints a JSON block containing every captured figure. That's the whole deliverable — paste it back and the corresponding # UNVERIFIED blocks and checklist items across the article can be filled in and ticked.

---

Where It Bites You

Not selecting the T4 runtime. The default Colab runtime is CPU. Cell 0 catches it, which is why it's cell 0.

Skipping the restart after install. The kernel is still holding the old torch, and you'll get an import error or an undefined symbol that looks far more alarming than it is.

Running the offline and server sections without freeing memory in between. Both allocate a KV cache pool at 90% of the card. Cell 7 does del llm plus gc.collect() and torch.cuda.empty_cache() for exactly this reason — if you run cells out of order you'll get an OOM that has nothing to do with your model.

Colab disconnecting mid-run. Free-tier sessions time out, and cell 9 in particular takes a while because it starts four servers in sequence. Run the short list above if you're on a flaky connection, and copy the results JSON out as you go rather than at the end.

Expecting these numbers to describe your production hardware. A T4 is vLLM's minimum supported GPU. The shapes transfer — batching helps, TTFT scales with prompt length, preemption inverts throughput — but the absolute figures belong to this card and this 0.5B model.

Treating a failed cell as a broken article. Cell 6 in particular may fail on a transformers API change rather than an arithmetic error — past_key_values has been migrating to a Cache object. The cell catches that and records it as a failure mode rather than crashing, because "the accessor changed" is itself a useful finding.

---

Cheat Sheet

Run it

Runtime → Change runtime type → T4 GPU → Run all
(restart when prompted at cell 2, then continue from cell 3)

Minimum useful subset

Cell	Settles
0 + 1	Hardware is adequate; capacity predicted
3	Predicted vs actual blocks — the overhead term
6	Whether the KV formula is exact
9	Whether max_num_seqs makes preemption worse
10	Prints everything captured

Expected interruptions

Moment	What to do
Cell 2 prompts a restart	Take it, resume at cell 3
Cell 3 asks for BLOCKS	Paste from the log, re-run the cell
Cell 7 frees the offline engine	Don't skip it — one GPU, one KV pool

The output that matters: cell 10's JSON block.

---

← Previous: Streaming & Client Patterns · Next: Stage 4 — Scaling & Performance →

---

⚠️ Verification checklist (delete before publishing)

The notebook itself has not been run. It is assembled from the per-page snippets, which are individually unverified. Running it end to end on a Colab T4 is the single highest-value action remaining for the whole article.

Known risks in the notebook, in order of likelihood

• Cell 9 (preemption) is the most fragile. It starts four servers in sequence, each taking up to three minutes, and counts preempt occurrences in captured stdout. Risks: the log may not reach stdout as captured; p.communicate() after terminate() may lose buffered output; the whole cell may exceed a free-tier session. Consider splitting it into four cells.
• Cell 3's contextlib.redirect_stdout may not capture vLLM's logs at all, since they go through logging to stderr from a subprocess, not Python-level stdout. If the log comes back empty, fall back to running vllm serve as a subprocess and reading its pipe.
• Cell 6 (past_key_values) is guarded with try/except but the accessor may need rewriting for the current transformers.
• Cell 7's del llm may not free GPU memory reliably in a notebook, where references linger in Out[] history. May need %reset -f or a runtime restart between the offline and server sections.
• The render endpoint call in cell 7 is unverified — payload shape is constructed by analogy.
• HfApi().model_info(..., files_metadata=True) requires network access to the Hub; confirm it works from Colab without a token for public models.
• Cell 5 estimates the looped time from 8 prompts ×4 rather than running all 32, to save time. State that inline in the notebook so the figure isn't mistaken for a direct measurement.

Consistency

• dtype= standardised across the article and the notebook. torch_dtype= is deprecated in transformers in favour of dtype=; Stage 0 page 1 was the outlier and has been fixed.
• Two broken links found and fixed — cells 17 and 23 referenced 01-concepts/... without the ../ prefix. All notebook markdown links now resolve, verified against the filesystem.

After a successful run

• Replace every # UNVERIFIED block across the article with captured output.
• Update VERIFICATION to mark the notebook as the canonical way to run sessions 1 and 2, superseding the per-page instructions there.
• Record the prediction error from cell 3 in Anatomy as the stated accuracy of the capacity script.