Most ML teams test their serving API and nothing else. Then a refactor silently detaches a layer from the graph, a loss still goes down a little, and the regression is discovered three weeks later in a metrics review. PyTorch code is ordinary Python, and it can be tested cheaply on CPU in seconds. This tutorial lays out a test pyramid we use on client projects.
The pyramid
- Contract tests (milliseconds): shapes, dtypes, device handling, no NaNs.
- Gradient tests (milliseconds): every trainable parameter receives a gradient.
- Behavior tests (seconds): the model can overfit one tiny batch; checkpoints round-trip.
- Regression gates (minutes, nightly): a fixed small eval set with a metric floor.
All of levels 1 to 3 should run on CPU with tiny configs, so they fit in any CI runner.
A model worth testing
# model.py
import torch
from torch import nn
class TinyClassifier(nn.Module):
def __init__(self, d_in=16, d_hidden=32, n_classes=4):
super().__init__()
self.net = nn.Sequential(
nn.Linear(d_in, d_hidden), nn.ReLU(), nn.Linear(d_hidden, n_classes)
)
def forward(self, x):
return self.net(x)
Design for testability: build models from a small config so a test can instantiate a 10k-parameter version of the same architecture you train at scale.
1. Contract tests
# test_contracts.py
import pytest, torch
from model import TinyClassifier
@pytest.mark.parametrize("batch", [1, 2, 7])
def test_output_shape_and_dtype(batch):
model = TinyClassifier().eval()
out = model(torch.randn(batch, 16))
assert out.shape == (batch, 4)
assert out.dtype == torch.float32
assert torch.isfinite(out).all()
def test_batch_independence():
# In eval mode, a sample's output must not depend on its batch neighbours.
model = TinyClassifier().eval()
x = torch.randn(5, 16)
with torch.no_grad():
full = model(x)
single = torch.stack([model(x[i:i+1])[0] for i in range(5)])
torch.testing.assert_close(full, single, rtol=1e-5, atol=1e-6)
The batch-independence test catches a surprising number of bugs: leaking batch-norm statistics, masks applied across the wrong axis, and attention that mixes sequences.
2. Gradient tests
def test_all_parameters_receive_gradients():
torch.manual_seed(0)
model = TinyClassifier().train()
x, y = torch.randn(8, 16), torch.randint(0, 4, (8,))
loss = torch.nn.functional.cross_entropy(model(x), y)
loss.backward()
for name, p in model.named_parameters():
assert p.grad is not None, f"{name} has no gradient"
assert torch.isfinite(p.grad).all(), f"{name} has non-finite gradient"
assert p.grad.abs().sum() > 0, f"{name} gradient is all zeros"
This is the test that catches the detached layer, the stray .detach() or .item(), and the parameter you forgot to register (a plain tensor attribute instead of nn.Parameter, or a layer kept in a Python list instead of nn.ModuleList).
For custom autograd functions or hand-written ops, use torch.autograd.gradcheck with double precision inputs:
from torch.autograd import gradcheck
def test_custom_op_gradcheck():
x = torch.randn(3, 4, dtype=torch.double, requires_grad=True)
assert gradcheck(my_custom_op, (x,), eps=1e-6, atol=1e-4)
3. Behavior tests
Overfit one batch
A correct model with a correct loss and optimizer can memorize 8 examples. If it cannot, something is broken in the model, loss, labels, or optimizer wiring.
def test_can_overfit_one_batch():
torch.manual_seed(0)
model = TinyClassifier().train()
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
x, y = torch.randn(8, 16), torch.randint(0, 4, (8,))
first = None
for step in range(200):
opt.zero_grad()
loss = torch.nn.functional.cross_entropy(model(x), y)
loss.backward()
opt.step()
first = first if first is not None else loss.item()
assert loss.item() < 0.05 * first
Checkpoint round-trip
def test_checkpoint_roundtrip(tmp_path):
torch.manual_seed(0)
a = TinyClassifier().eval()
path = tmp_path / "ckpt.pt"
torch.save(a.state_dict(), path)
b = TinyClassifier().eval()
b.load_state_dict(torch.load(path, weights_only=True))
x = torch.randn(4, 16)
with torch.no_grad():
torch.testing.assert_close(a(x), b(x))
Using weights_only=True also keeps you on the safer loading path. Load with the default strict=True so a renamed layer fails loudly instead of silently leaving random weights.
Determinism
def test_same_seed_same_result():
def run():
torch.manual_seed(123)
m = TinyClassifier()
return m(torch.randn(4, 16))
torch.testing.assert_close(run(), run())
For GPU determinism across kernels, see our post on reproducible PyTorch training.
4. Regression gates
Keep a small, frozen evaluation set (a few hundred examples) in the repo or in artifact storage, and a nightly job that trains or loads a reference checkpoint and asserts a floor:
@pytest.mark.nightly
def test_metric_floor():
acc = evaluate(load_reference_model(), load_frozen_eval_set())
assert acc >= 0.92, f"accuracy regressed to {acc:.3f}"
Set the floor a little below the current value to absorb noise, and ratchet it up deliberately when the model improves. Register the marker in pyproject.toml and exclude it from the per-commit run:
[tool.pytest.ini_options]
markers = ["nightly: slow regression gates"]
addopts = "-m 'not nightly'"
Practical tips
- Always use
torch.testing.assert_close, not==orallcloseby hand; it checks dtype, shape and gives readable diffs. - Test in both
train()andeval()modes for anything with dropout or batch norm. - Parametrize over dtypes (float32, bfloat16) with looser tolerances for lower precision.
- Guard GPU tests with
@pytest.mark.skipif(not torch.cuda.is_available(), reason="needs GPU")and run them on a scheduled GPU runner rather than every commit. - Keep tiny configs in a fixture so every test builds the same small model quickly.
- Seed explicitly inside each test; never rely on global state from another test.
A minimal CI layout
- Every pull request: contract, gradient, and behavior tests on CPU (under a minute).
- Nightly: regression gates and GPU-marked tests.
- Before release: the regression gate plus a latency/memory budget check.
Where this pays off
These tests are cheap to write and make refactors, PyTorch version upgrades and torch.compile adoption far less risky, because you can tell within seconds whether behavior changed. If you are inheriting an untested training codebase or planning a migration, IntelliSensei can help build this safety net first. Get in touch via our contact page.