Background
Back to Articles/PyTorch 101: Zero to Hero
Sections
IntroductionModule 01 — Tensors 🧊01 · Creating Tensors 🧊02 · Indexing & Reshaping 🔪03 · Tensor Math ➗04 · Device Placement 🖥️⚡Module 02 — Autograd ⚙️01 · The Computational Graph 🕸️02 · Backward Pass & Gradients ⬅️03 · Turning Autograd Off 🛑04 · Gradient Gotchas 🪤Module 03 — Neural Networks01 · The `nn.Module` Basics 🧱02 · Common Layers 🧩03 · Building a Network 🏗️04 · Inspecting Models 🔍Module 04 — Data Handling 🗂️01 · Dataset Basics 📇02 · The DataLoader 🚚03 · Transforms 🎨04 · Splits & Built-in Datasets ✂️Module 05 — The Training Loop 🔁01 · Loss Functions 🎯02 · Optimizers ⚙️03 · The Training Loop 🔁04 · Evaluation & Metrics 📊Module 06 — Saving & Loading 💾01 · `state_dict` Basics 💾02 · Checkpoints & Resuming ⏸️03 · Loading for Inference 🚀04 · Best Model & Early Stopping 🏅Module 07 — Computer Vision 👁️01 · Convolutions 🔲02 · CNN Architecture 🏛️03 · Transfer Learning 🔁04 · Image Classification Project 🧪Module 08 — NLP & Transformers 💬01 · Text Data & Tokenization 🔤02 · Embeddings 🧭03 · Recurrent Layers & LSTMs 🔄04 · Intro to Transformers ⚡Module 09 — Ecosystem: PyTorch Lightning ⚡01 · Why Lightning? 🤔02 · The LightningModule 🧩03 · The Trainer 🎛️04 · DataModules & Callbacks 🧰Module 10 — Deployment & Optimization 🚀01 · Exporting Models 📦02 · `torch.compile` ⚡03 · Inference Optimization 🪶04 · Serving & Next Steps 🎓

01 · Creating Tensors 🧊

4 min read

Before you can do anything in PyTorch, you need numbers in a tensor. Let's learn every way to put them there.

1. The 'Why'

Every deep learning framework needs a way to store numbers — lots of them. An image is millions of pixel values; a digitized sentence is a grid of numbers; a neural network's weights are enormous tables of decimals. All of it lives inside tensors, so the very first skill you need is simply creating them. It sounds trivial, but the choices you make at creation time — the shape, and especially the data type — ripple through everything that follows. Get them wrong and you'll hit cryptic errors or silently slow training much later.

Why does PyTorch have its own tensor instead of reusing the NumPy array you may already know? Two reasons that are the whole reason PyTorch exists. First, tensors can live on a GPU, where thousands of tiny operations run in parallel — the difference between training in an hour versus a week. Second, tensors plug into PyTorch's autograd engine (Module 02), which tracks operations so gradients can be computed automatically. NumPy can do neither. So while a tensor looks like a NumPy array, it's built for the specific demands of training neural networks — and it all starts with knowing how to create one.

2. Core Concepts

What a tensor is. A tensor is an n-dimensional grid of numbers. The number of dimensions is its rank. A helpful analogy is nested Excel spreadsheets: a scalar is a single cell, a vector is one row, a matrix is a full sheet, a 3-D tensor is a workbook of sheets, and 4-D and beyond are stacks of stacks (e.g. a batch of color images is [batch, channels, height, width]).

The three attributes you'll check constantly. Every tensor carries shape (size along each dimension), dtype (the element data type, e.g. torch.float32), and device (where its memory lives — covered in sub-module 04).

Data types matter. The default for decimals is torch.float32, which is what neural networks almost always want. Integers default to torch.int64. Choosing the right dtype controls memory use and numerical precision, and mismatched dtypes are a common source of errors.

3. Code in Action

Creating from existing data

import torch

# From a Python list — dtype is inferred automatically from the values
a = torch.tensor([[1.0, 2.0, 3.0],
                  [4.0, 5.0, 6.0]])       # a 2x3 matrix of floats

print(a)                 # show the tensor's contents
print(a.shape)           # torch.Size([2, 3]) -> 2 rows, 3 columns
print(a.dtype)           # torch.float32 -> default floating-point type
print(a.ndim)            # 2 -> the rank (number of dimensions)

Factory functions for common initializations

import torch

zeros  = torch.zeros(2, 3)         # a 2x3 tensor filled entirely with 0.0
ones   = torch.ones(2, 3)          # a 2x3 tensor filled entirely with 1.0
rand   = torch.rand(2, 3)          # a 2x3 tensor of random floats in [0, 1)
arange = torch.arange(0, 10, 2)    # 1-D tensor: [0, 2, 4, 6, 8] (start, stop, step)
like   = torch.zeros_like(rand)    # zeros with the SAME shape/dtype/device as `rand`

Controlling the data type

import torch

# Set a dtype explicitly at creation time
ints = torch.tensor([1, 2, 3], dtype=torch.int64)   # force 64-bit integers
half = torch.ones(2, 2, dtype=torch.float16)         # half precision (saves memory)

# Cast an existing tensor to a new dtype
x = torch.tensor([1, 2, 3])        # inferred as int64
x_float = x.float()                # convert to float32 (returns a NEW tensor)
print(x.dtype, x_float.dtype)      # torch.int64 torch.float32

# Reproducible randomness — seed the generator first
torch.manual_seed(42)              # makes torch.rand outputs deterministic
print(torch.rand(3))               # same three numbers every run

4. Common Pitfalls

Pitfall 1 — Integer tensors where you meant floats. torch.tensor([1, 2, 3]) gives you int64, but neural networks and most math expect float32. Feeding integers into a model throws dtype errors or silently misbehaves. Fix: write 1.0 instead of 1, or call .float().

Pitfall 2 — Forgetting to seed randomness. If you use torch.rand/torch.randn without torch.manual_seed(...), your results change every run, making bugs impossible to reproduce. Seed at the top of any experiment.

Pitfall 3 — torch.Tensor vs torch.tensor. The lowercase torch.tensor(...) (recommended) infers dtype from your data. The capital-T torch.Tensor(...) is a constructor that treats a single integer as a size, so torch.Tensor(3) gives an uninitialized length-3 tensor of garbage values, not tensor(3). Prefer lowercase.

5. Further Reading & Watch List

➡️ Next: 02 · Indexing & Reshaping