We are called in when a PyTorch project is late, over budget, or producing a model that works in a notebook and nowhere else. The framework is rarely the problem. The same handful of organizational and engineering mistakes account for most of what we see, and each has a known fix. Here are the ones that recur most.
1. Eager-mode performance habits
PyTorch's eager mode is wonderful for development and it trains people to write code that is slow in production: Python loops over batch elements, .item() calls in the training loop that force GPU synchronization, hand-written attention, FP32 everywhere, data loading on the main thread. Teams then buy more GPUs instead of fixing the code.
The fix. Profile before scaling. torch.profiler will show within an hour whether the GPU is waiting on Python, on data, or on synchronization. BF16 autocast, scaled_dot_product_attention, a fused optimizer and torch.compile routinely deliver 2-4x on models that have never been profiled, at no hardware cost. See torch.compile in practice.
2. Unversioned training data
The model is versioned in git and the checkpoint is in object storage, but the training set was a query somebody ran against the warehouse eight months ago. Nobody can reproduce the model, so nobody can improve it safely, and when a regulator or a customer asks what it was trained on the answer is a shrug.
The fix. Treat data as a build input. Snapshot training and evaluation sets to immutable storage with a content hash, record the hash in the training run's metadata, and make the evaluation set a protected artifact that training code cannot read. Decontaminate training data against it. This is a week of engineering and it pays for itself on the first "why did the model change?" investigation.
3. Serving as an afterthought
The model hits its accuracy target, and then the team discovers it takes two seconds per request, does not fit on the inference GPUs that were budgeted, or depends on a preprocessing pipeline that only exists in a notebook. Months of work sit unshipped while serving gets retrofitted.
The fix. Decide the serving target at the start (vLLM, Triton, LitServe, ExecuTorch, or plain FastAPI) and set latency and cost budgets alongside the accuracy target. Export the model (torch.export) and run it through the serving path in week one with random weights; the model can improve while the pipeline already works. Teams still on TorchServe, which was archived in 2025, have a specific version of this problem; see Migrating off TorchServe.
4. Evaluating on the training distribution
Accuracy is 97% in testing and 70% in production. The test set was a random split of the same scrape, so it shared duplicates, templates and time period with training; production data is next month's.
The fix. Split by time and by entity (customer, document, device), not at random. Deduplicate across the split. Keep a small, hand-labelled "golden" set drawn from real production inputs and report on it separately. For LLM work, add an LLM-as-judge rubric with human spot checks, and re-run evaluation on every model change in CI.
5. Fine-tuning when retrieval was the answer, or the reverse
Enterprises either fine-tune an LLM to "teach it our documents" and are surprised when it still hallucinates, or build a RAG pipeline to change the model's tone and format and are surprised when retrieval does not help. These are different tools for different problems.
The fix. Fine-tuning changes behaviour (format, style, task skill); retrieval supplies knowledge (facts, documents, freshness). Most real systems need retrieval first and a modest fine-tune second. Our decision framework walks through it.
6. Pinned to PyTorch 1.x
A working 1.13 environment on CUDA 11 feels safe until a new GPU arrives that the driver does not support, or a library you need drops 1.x. Then the upgrade happens under pressure, on a deadline, with no regression harness.
The fix. Upgrade on your schedule, not the hardware's. Build the golden-output regression harness first, then upgrade the toolchain, then the code. The 1.x to 2.x checklist is the sequence we use; it takes days when planned and weeks when forced.
7. No one owns the model after launch
The data scientist who built it moved to another project; the platform team that runs it does not know what the metrics mean. Drift goes unnoticed for months until a business metric moves.
The fix. Name an owner, define the two or three metrics that would indicate a problem (input distribution shift, confidence collapse, downstream KPI), alert on them, and schedule a quarterly retrain-or-retire review. This is routine maintenance, not research, and it can be handled under a support retainer.
The pattern
None of these are deep-learning problems. They are software-engineering and ownership problems that deep learning makes expensive. The teams that ship reliably are the ones that treat a model like any other production component: versioned inputs, a regression suite, a deployment target chosen up front, and a named owner. If your project has stalled on one of these, that is exactly the work our project rescue engagements take on. Contact us to talk it through.