Module 08 — NLP & Transformers 💬
Images are grids of numbers; text is a sequence of symbols. This module teaches PyTorch to read — from turning words into vectors to the architecture behind modern language models.
In Module 07 you handled images, where the data arrives as neat numeric grids. Language is fundamentally different, and that difference is what makes Natural Language Processing (NLP) its own discipline. Text is a sequence of discrete symbols — words or sub-words — with no inherent numeric meaning, of variable length, where order carries enormous significance ("dog bites man" versus "man bites dog"). Before a neural network can touch it, you must convert raw strings into numbers, and you need architectures that respect sequence and context. This module walks that path: first the plumbing of getting text into tensors, then the representations and models that let a network actually understand it.
The journey mirrors the field's own evolution. You'll start by tokenizing text and looking words up in a vocabulary, then learn embeddings — dense, learnable vectors that capture meaning far better than crude one-hot codes. From there you'll meet recurrent networks (LSTMs), which read a sequence one step at a time while carrying a memory, the workhorse of NLP for years. Finally you'll get an introduction to the Transformer — the attention-based architecture that replaced recurrence and powers today's large language models. By the end you'll understand the arc from raw text to the models making headlines.
This module is split into three sub-modules — work through them in order.
📚 Sub-Modules
| # | Sub-Module | What you'll learn |
|---|---|---|
| 01 | Text Data & Tokenization | Tokenizing text, building a vocabulary, numericalizing, and padding sequences into batches |
| 02 | Embeddings | nn.Embedding and why dense learnable vectors beat one-hot encodings |
| 03 | Recurrent Layers & LSTMs | RNNs, the vanishing-gradient problem, and nn.LSTM for sequences |
| 04 | Intro to Transformers | The attention idea, why Transformers replaced RNNs, and nn.Transformer basics |
🎯 By the end of this module, you'll be able to...
- Convert raw text into padded tensors a model can consume.
- Represent tokens with learnable embeddings instead of sparse one-hot vectors.
- Build a sequence model with
nn.LSTMand explain why LSTMs beat vanilla RNNs. - Explain self-attention and recognize the building blocks of
nn.Transformer.
✅ Prerequisites
All of Phases 1–2, plus comfort with building models (Module 03) and the data pipeline (Module 04). The collate_fn/padding idea from Module 04's DataLoader sub-module returns here.
⬅️ Prev module: 07 · Computer Vision · ➡️ Next module: 09 · Ecosystem: Lightning