Module 1 · PyTorch Deep Learning

Module 1: Introduction & PyTorch Tensors

Master PyTorch tensors. Learn basic initializations, dynamic computation graphs vs. static graphs, and memory management.

⏱ 20 Min Read • Author: GenAIWallah Team • Updated: May 2026

Day 1

Tensors in PyTorch

Why this matters

Tensors: Tensors are the fundamental data structure — every image, batch, and weight matrix is a tensor with shape and dtype.

Tensors generalize scalars, vectors, and matrices to arbitrary dimensions. In PyTorch, almost everything is a torch.Tensor.

Shape: e.g. (batch, channels, height, width) for images.
dtype: float32 is default for neural nets; int64 for labels.
device: CPU or CUDA — set when creating or moving tensors.

import torch

x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
print(x.shape)   # torch.Size([2, 2])
print(x.dtype) # torch.float32

y = torch.zeros(3, 4)
z = torch.randn(2, 3)  # standard normal

NumPy interop: torch.from_numpy(arr) shares memory when possible; clone if you need independence.

Common mistakes

Forgetting optimizer.zero_grad() so gradients accumulate across batches.
Tensor shape mismatches (especially batch/channel dimensions for CNNs).
Training on GPU but leaving tensors on CPU (or vice versa).

Interview checkpoints

Q: Explain tensors in PyTorch. A: One-sentence definition + shape/device note.
Q: Common bug? A: Gradients, shapes, or device mismatch.

Practice

Basic: Define Tensors and sketch a minimal code snippet.
Intermediate: Run a notebook cell demonstrating Tensors.
Advanced: Intentionally break Tensors and interpret the error.

Recap

You can explain tensors clearly.
You know one mistake to avoid.
You see how this connects to the next lesson.

Next: Dynamic Graphs

Day 2

Dynamic vs. Static Computation Graphs

Why this matters

Dynamic Graphs: PyTorch builds the graph dynamically (define-by-run), which simplifies debugging and control flow vs static graphs.

PyTorch uses a dynamic (define-by-run) graph: the forward pass builds the computation graph on the fly. TensorFlow 1.x used static graphs; PyTorch 2.x can compile but defaults to eager mode.

Dynamic (PyTorch)	Static (classic TF)
Python control flow in forward	Graph built then executed
Easier debugging	Harder mid-graph prints
Graph rebuilt each forward (unless compile)	Single optimized graph

Common mistakes

Forgetting optimizer.zero_grad() so gradients accumulate across batches.
Tensor shape mismatches (especially batch/channel dimensions for CNNs).
Training on GPU but leaving tensors on CPU (or vice versa).

Interview checkpoints

Q: Explain dynamic graphs in PyTorch. A: One-sentence definition + shape/device note.
Q: Common bug? A: Gradients, shapes, or device mismatch.

Practice

Basic: Define Dynamic Graphs and sketch a minimal code snippet.
Intermediate: Run a notebook cell demonstrating Dynamic Graphs.
Advanced: Intentionally break Dynamic Graphs and interpret the error.

Recap

You can explain dynamic graphs clearly.
You know one mistake to avoid.
You see how this connects to the next lesson.

Next: Autograd

← Back to PyTorch Hub Module 2: Autograd Engine →