Module 05 — The Training Loop 🔁
This is the moment everything converges. Tensors, autograd, models, and data all come together into the loop that actually makes a network learn.
Every module so far has been building toward this one. You have tensors (01) and the autograd engine that differentiates them (02); you can build a model out of layers (03); and you can serve it batched, shuffled data (04). The training loop is the conductor that brings them together: for each batch, it runs a forward pass to get predictions, measures how wrong they are with a loss function, uses autograd to compute gradients, and asks an optimizer to nudge every weight in the direction that reduces the loss. Repeat that a few thousand times and a random pile of numbers becomes a working model.
The loop itself is short — often a dozen lines — but every line earns its place, and the order matters. Zero the old gradients, forward, compute loss, backward, step: get the sequence wrong and training silently breaks (remember gradient accumulation from Module 02?). This module also covers the two modes a model can be in — train() and eval() — and why every serious training run alternates between learning on the training set and measuring honestly on a validation set. Master this loop once and you'll reuse the exact same skeleton for CNNs, Transformers, and everything else in Phase 3.
This module is split into three sub-modules — work through them in order.
📚 Sub-Modules
| # | Sub-Module | What you'll learn |
|---|---|---|
| 01 | Loss Functions | What a loss measures, and choosing MSELoss vs. CrossEntropyLoss for your task |
| 02 | Optimizers | SGD vs. Adam, the learning rate, and the zero_grad() → step() rhythm |
| 03 | The Training Loop | Assembling the full loop, backprop, and model.train() vs. model.eval() |
| 04 | Evaluation & Metrics | A proper validation loop, computing accuracy, and reading curves to spot overfitting |
🎯 By the end of this module, you'll be able to...
- Pick and apply the right loss function for regression or classification.
- Configure an optimizer and explain what the learning rate controls.
- Write a complete, correct training loop with a validation pass — from scratch.
- Evaluate a model with accuracy metrics and diagnose overfitting from train/val curves.
✅ Prerequisites
All of Phase 1, plus the data pipeline from Module 04. The gradient-accumulation and train()/eval() ideas from Module 02 and Module 03 become concrete here.
⬅️ Prev module: 04 · Data Handling · ➡️ Next module: 06 · Saving & Loading