Free through lesson 3

PyTorch & Deep Learning for Beginners

From tensors and automatic differentiation to neural network basics, techniques for stable training, CNNs and transfer learning, Attention and Transformers, faster inference, and shipping your model. Across 50 lessons, you'll get to the point where you can build a model's internals yourself, train it, and ship it as ONNX and an API. Every piece of code and output was actually run on PyTorch 2.14.0.

Start from lesson 1 Unlock every course

Curriculum

The 50 lessons are split into 7 chapters. We recommend going in order from chapter 1, but feel free to jump to whatever interests you. Note: The lessons use torch, so they can't run in the browser. Create a virtual environment on your machine, run pip install torch torchvision, and try things there. You don't need a GPU. If CUDA is available you can use cuda, and on an Apple Silicon Mac you can use mps.

Chapter 1 — Tensors and Automatic Differentiation (lessons 1–8)

You'll get a handle on the two foundations of PyTorch: tensors and automatic differentiation. At the end, you'll solve linear regression with plain gradient descent, without an optimizer.

1

Where Deep Learning and PyTorch Fit

Sort out what this technology automates, set up your environment, and check which device works on your machine.

Free
2

Creating Tensors: shape and dtype

Almost the same tool as a NumPy array. The differences are dtype and the fact that from_numpy shares memory.

Free
3

Operations, Reshaping, and Broadcasting

* is elementwise, @ is matrix multiplication. The difference between view and reshape, and how to read shape-mismatch errors.

Free
4

Reductions Along an Axis and Indexing

Specify dim and that axis collapses. Check what argmax, keepdim, and squeeze mean by looking at shapes.

🔒 Basic
5

Switching Devices (CPU, MPS, CUDA)

Move tensors with to(device); you can only compute between tensors on the same device. You'll also measure how a GPU can be slower on small computations.

🔒 Basic
6

autograd: How Gradients Are Computed Automatically

Just set requires_grad and call backward. Plus how gradients accumulate, and what to do about it.

🔒 Basic
7

no_grad, detach, and Cutting the Computation Graph

Don't build gradients during inference. Three ways to cut the graph, and when each one fits.

🔒 Basic
8

Writing Gradient Descent by Hand

Solve linear regression with just w -= lr * w.grad, no optimizer. Including what divergence looks like when the learning rate is too large.

🔒 Basic

Chapter 2 — Neural Network Basics (lessons 9–16)

You'll run fully connected layers, activation functions, loss functions, nn.Module, the training loop, and DataLoader. At the end, you'll write one complete binary classifier, from standardization all the way to a confusion matrix.

9

Inside the Fully Connected Layer nn.Linear

weight is ordered (output, input). Confirm it matches x @ W.T + b, and count the parameters by hand.

🔒 Basic
10

Why Activation Functions Are Needed

Stack any number of linear layers and you still get one layer. Measure the values of ReLU, Sigmoid, and GELU, and how gradients vanish.

🔒 Basic
11

The Task Decides the Loss Function

MSE for regression, CrossEntropy for multiclass, BCEWithLogits for binary. Including the accident of applying softmax twice.

🔒 Basic
12

Defining a Model with nn.Module

Build the parts in __init__ and connect them in forward. Plus the pitfall of layers in a list not getting registered.

🔒 Basic
13

The 5 Steps of the Training Loop

zero_grad → forward → loss → backward → step. This order stays the same for CNNs and Transformers alike.

🔒 Basic
14

Dataset and DataLoader

The machinery that feeds mini-batches. Leftover batches and drop_last, and the 3 methods of a custom Dataset.

🔒 Basic
15

train vs. eval: Rules for Evaluation

Dropout and BatchNorm behave differently in training and inference. Forget to switch and your accuracy wobbles.

🔒 Basic
16

Running Your First Classifier End to End

Standardization, devices, the training loop, and a confusion matrix in one script, using only the tools from chapters 1 and 2.

🔒 Basic

Chapter 3 — Stabilizing Training (lessons 17–24)

You'll compare learning rates, optimizers, schedulers, initialization, regularization, Dropout, and normalization layers, all on the same data, and check the numbers. At the end, you'll split off validation data and implement early stopping.

17

How to Choose a Learning Rate

Too small and it doesn't move; too large and it diverges. Sweep 5 orders of magnitude, measure, and confirm the divergence condition with math.

🔒 Basic
18

Choosing an Optimizer (SGD, Momentum, Adam)

Adam adapts the learning rate per parameter. Compare them at the same lr, and look at what each one keeps as state.

🔒 Basic
19

Learning Rate Schedulers

How StepLR, Cosine, and ReduceLROnPlateau work. You'll also measure that adding one doesn't always help.

🔒 Basic
20

Weight Initialization and Vanishing Gradients

All zeros and training never starts. Measure what standard deviations Xavier and Kaiming produce, and how far the gradients reach.

🔒 Basic
21

Keeping Weights Small with Weight Decay

With an extreme setup of 67,074 parameters and 120 training samples, see in numbers how regularization works.

🔒 Basic
22

Dropout

Drop neurons only during training. Look at the values to see why the 1/(1-p) scaling is needed.

🔒 Basic
23

Batch Normalization and LayerNorm

BatchNorm works across the batch, LayerNorm per sample. Including why it fails with a batch size of 1.

🔒 Basic
24

Validation Data and Early Stopping

Never decide when to stop by looking at the test set. Early stopping includes rolling back to the weights with the lowest validation loss.

🔒 Basic

Chapter 4 — Images (lessons 25–32)

From how convolution and pooling work to residual connections, data augmentation, and transfer learning. At the end, you'll fine-tune a pretrained resnet18 and measure how it compares to training from scratch.

25

Image Data Shapes and Normalization

The order is (N, C, H, W). ToTensor scales to 0 to 1, and Normalize centers around a mean of 0.

🔒 Basic
26

How Convolution (Conv2d) Works

Reuse a small filter across the whole image. The output-size formula, and a 50,000x difference from a fully connected layer.

🔒 Basic
27

Pooling and Receptive Fields

Halve the height and width and become robust to small shifts. Count how far the receptive field grows as you stack layers.

🔒 Basic
28

Training a Small CNN on MNIST

6,000 images, 5 epochs, 5 seconds. A CNN with one fifth the parameters beats a fully connected network on accuracy.

🔒 Basic
29

Residual Connections and Well-Known Architectures

Just adding x + f(x) makes depth pay off. Stack up to 16 blocks and confirm it by measurement.

🔒 Basic
30

Data Augmentation

Pad out your training data with flips, crops, and rotations. You'll also measure that it sometimes doesn't help.

🔒 Basic
31

Transfer Learning: Borrowing Pretrained Features

Just putting a linear classifier on top of resnet18 features learned on ImageNet far outperforms training from scratch.

🔒 Basic
32

Fine-Tuning and Freezing Layers

How much to unfreeze, and what learning rate to use. Multiply lr by 100 and the pretrained weights get destroyed.

🔒 Basic

Chapter 5 — Sequences and Attention (lessons 33–42)

You'll measure the limits of RNNs, write Attention yourself, and assemble a Transformer block. At the end, you'll train a character-level language model on your own Transformer and have it generate sentences.

33

Sequence Data Shapes and Padding

Three dimensions: (batch, time, features). When to use masks vs. pack to align sequences of different lengths.

🔒 Basic
34

Embeddings with nn.Embedding

Same result as one-hot times a matrix, but faster since it just looks up a row. Including what padding_idx means.

🔒 Basic
35

How RNNs Work and Their Limits

The same weights are reused at every time step. Show, in orders of magnitude, how gradients vanish the further back in time they go.

🔒 Basic
36

LSTM and GRU

Add gates so the model learns what to remember and what to forget. Measure the difference against an RNN under the same conditions.

🔒 Basic
37

Writing Attention by Hand

Just softmax the Q·K dot products and mix V. Confirm with numbers why we divide by sqrt(d).

🔒 Basic
38

Multi-Head Attention

Split the dimensions and run attention in parallel. Changing the number of heads doesn't change the parameter count.

🔒 Basic
39

Positional Encoding

Attention alone can't tell word order. The sinusoidal formula vs. learned position embeddings.

🔒 Basic
40

Assembling a Transformer Block

Attention → FFN, each with a residual connection and LayerNorm. Pre-LN vs. Post-LN changes how gradients flow by orders of magnitude.

🔒 Basic
41

Predicting Characters with Your Own Transformer

In 600K parameters and 35 seconds, teach it the grammar of a corpus generated from templates.

🔒 Basic
42

Using the Built-In Transformer

nn.TransformerEncoderLayer has the same internals as yours. Just watch out for the mask types and the norm_first default.

🔒 Basic

Chapter 6 — Real-World Practice (lessons 43–47)

Experiment logging, reproducibility, faster inference, saving and loading, TorchScript and torch.compile. You'll gather the tools you need to reproduce the same results again and to make models fast enough for production.

43

Keeping a Record of Training

One CSV and one config file, so you can later say which settings produced which numbers.

🔒 Basic
44

Reproducibility and Debugging

What fixing the seed covers, and a 3-point check of shape, dtype, and gradients. Plus the trap of broadcasting that's silently wrong.

🔒 Basic
45

Making Inference Faster

Measure inference_mode, batching, and float16. Sending one item at a time is 33x slower.

🔒 Basic
46

Saving and Loading

Save the state_dict. The risks of saving the whole model, and what to put in a checkpoint.

🔒 Basic
47

TorchScript, Quantization, and torch.compile

Three ways to run without Python. The trap of trace dropping branches, and environment-dependent quantization.

🔒 Basic

Chapter 7 — Finishing Up (lessons 48–50)

You'll export to ONNX, wrap the model in an inference API with FastAPI, and get it into a shape you can ship. Finally, you'll fold the tools from all 50 lessons into a single script and sort out where to go next.

48

Exporting and Shipping with ONNX

Make it run outside PyTorch. What you ship is a set of three: the model, the preprocessing, and what the output means.

🔒 Basic
49

Turning a Trained Model into an API

One /predict endpoint with FastAPI. Load the model once at startup, and validate input shapes yourself.

🔒 Basic
50

Wrap-Up and Where to Go Next

Fold the tools from 50 lessons into one script, and finish by drawing a clear line between this course and Intro to AI App Development.

🔒 Basic

Once you've finished all 50 lessons, learn data preprocessing and classic methods in Python & Machine Learning for Beginners, and how to call large pretrained models and build apps on them in Intro to AI App Development. To put an inference API into production, head to Intro to Shipping and Running a Service. A membership unlocks every course.