IntelliSensei | Time-Series Forecasting in PyTorch: Zero-Shot Foundation Models vs a Model You Train

Every enterprise has a forecasting problem somewhere: demand planning, capacity, cash, churn, staffing, spare parts. Until recently the honest answer for most of them was "use gradient-boosted trees on lag features." In 2026 the landscape has changed: a family of pretrained time-series foundation models — Chronos-Bolt, TimesFM, Moirai, Lag-Llama, TabPFN-TS — will produce a usable zero-shot forecast for a series it has never seen, with no training run at all.

That is genuinely useful, and it is also the single most over-applied idea we see in client forecasting projects. This tutorial walks through how to evaluate forecasting properly in PyTorch, how zero-shot foundation models compare to a task-specific model you train yourself, and when each one is the right call.

Start with the baseline, not the model

Forecasting is the area of ML where naive baselines most often win, and where people most often skip them. Before you load anything from the Hub, compute these:

Seasonal naive: predict the value from one season ago (last week's same weekday, last year's same month).
Drift / random walk: last value, optionally with a fitted trend.
Seasonal naive + ETS or ARIMA per series, if your series count is small enough.

If a foundation model cannot beat seasonal naive on your data, that is a result about your data — often it means your series are dominated by irreducible noise or by exogenous events the model cannot see (promotions, outages, price changes). No architecture fixes that; better features do.

Get the evaluation right first

More forecasting projects fail on evaluation than on modelling. Four rules:

1. Backtest with rolling origins, never a single split. Fit (or condition) at time t, predict t+1..t+H, roll forward, repeat. A single holdout window tells you about one month of history, not about your process.

def rolling_origins(n, context, horizon, stride):
    """Yield (train_end, test_slice) pairs for a rolling backtest."""
    t = context
    while t + horizon <= n:
        yield t, slice(t, t + horizon)
        t += stride

2. Scale-free metrics, aggregated correctly. MAPE explodes near zero and punishes under-forecasting asymmetrically. Prefer MASE (error relative to in-sample seasonal naive) or weighted quantile loss. Aggregate across series with a weighted mean — usually weighted by revenue or volume, because a 40% error on a tail SKU is not equal to a 4% error on your top one.

import torch

def mase(y_true, y_pred, y_insample, season=7):
    # scale = mean absolute seasonal-naive error on the training history
    scale = (y_insample[season:] - y_insample[:-season]).abs().mean()
    return ((y_true - y_pred).abs().mean() / scale.clamp(min=1e-8))

def weighted_quantile_loss(y_true, q_pred, quantiles):
    # q_pred: [H, Q]; quantiles: [Q]
    e = y_true.unsqueeze(-1) - q_pred
    ql = torch.maximum(quantiles * e, (quantiles - 1) * e)
    return 2 * ql.sum() / y_true.abs().sum().clamp(min=1e-8)

3. Forecast the distribution, not the point. Almost every business decision downstream of a forecast is a decision under uncertainty: safety stock, staffing buffers, credit limits. A p50 with no p10/p90 is nearly useless for those, and quantile loss is what you should be optimizing and reporting.

4. No leakage from the future. Covariates must be knowable at prediction time. Calendar features, holidays and planned promotions are fine. "Actual weather during the horizon" and "actual price charged" are not, unless you have a committed plan for them. This is the single most common silent bug in client forecasting code we are asked to review.

Zero-shot: a foundation model in ten lines

Chronos-Bolt is the pragmatic default in 2026: it is a T5-style encoder-decoder over patched, quantized series values, it is small (a few tens of millions of parameters), it runs fast on CPU, and it emits quantiles directly.

import torch
from chronos import BaseChronosPipeline

pipe = BaseChronosPipeline.from_pretrained(
    "amazon/chronos-bolt-base",
    device_map="cuda",
    torch_dtype=torch.bfloat16,
)

# context: [num_series, context_len] float tensor of history
quantiles, mean = pipe.predict_quantiles(
    context=context,
    prediction_length=28,
    quantile_levels=[0.1, 0.5, 0.9],
)
# quantiles: [num_series, 28, 3]

That is the whole inference path — no training loop, no hyperparameter sweep. Batch a few thousand series at once and it is fast enough to refresh nightly on a single GPU, or on CPU for smaller portfolios.

What zero-shot models are good at:

Cold-start series with little history.
Very wide portfolios (tens of thousands of series) where per-series models are unmaintainable.
A credible baseline in a day, which is often exactly what a stakeholder needs before funding a real project.

What they are structurally bad at:

Exogenous drivers. Most zero-shot checkpoints take the target history and little else. If your demand is driven by price and promotion, the model literally cannot see the thing that causes the variance you care about.
Hierarchical coherence. Forecasts won't sum from SKU to category to region.
Your idiosyncratic calendar. Fiscal weeks, regional holidays, plant shutdowns.
Non-standard loss. If over-forecasting costs 5x under-forecasting, you need to train against that asymmetry.

Task-specific: a patch-based forecaster you train

When covariates matter, train a global model — one network across all series, with series identity as an embedding. A PatchTST-style encoder is a strong, unfussy choice: chop the context window into patches, embed, run a transformer encoder, project to H x Q quantiles.

import torch, torch.nn as nn

class PatchForecaster(nn.Module):
    def __init__(self, context=336, horizon=28, patch=16, d=256,
                 n_layers=4, n_series=None, n_cov=0, n_quantiles=3):
        super().__init__()
        self.patch, self.horizon, self.nq = patch, horizon, n_quantiles
        n_patches = context // patch
        self.proj = nn.Linear(patch + n_cov * patch, d)
        self.pos = nn.Parameter(torch.randn(1, n_patches, d) * 0.02)
        self.ident = nn.Embedding(n_series, d) if n_series else None
        enc = nn.TransformerEncoderLayer(d, 8, 4 * d, batch_first=True,
                                         norm_first=True, dropout=0.1)
        self.enc = nn.TransformerEncoder(enc, n_layers)
        self.head = nn.Linear(d * n_patches, horizon * n_quantiles)

    def forward(self, y, cov=None, sid=None):
        # y: [B, context]  -> instance normalization (RevIN-style)
        mu = y.mean(-1, keepdim=True)
        sd = y.std(-1, keepdim=True).clamp(min=1e-5)
        z = ((y - mu) / sd).unfold(-1, self.patch, self.patch)   # [B, P, patch]
        if cov is not None:
            c = cov.unfold(-2, self.patch, self.patch).flatten(-2)
            z = torch.cat([z, c], dim=-1)
        h = self.proj(z) + self.pos
        if self.ident is not None and sid is not None:
            h = h + self.ident(sid).unsqueeze(1)
        h = self.enc(h).flatten(1)
        out = self.head(h).view(-1, self.horizon, self.nq)
        return out * sd.unsqueeze(-1) + mu.unsqueeze(-1)          # de-normalize

Two details carry most of the accuracy. Instance normalization (normalize each window by its own mean and std, then invert at the output) is what makes a single global model work across series whose scales differ by four orders of magnitude. And patching cuts sequence length by the patch size, so a 336-step context costs 21 tokens instead of 336 — quadratic attention stops mattering and you can afford long contexts.

Train it against quantile loss, which also gives you the intervals for free:

quantiles = torch.tensor([0.1, 0.5, 0.9], device="cuda")

def pinball(pred, target, q):
    e = target.unsqueeze(-1) - pred
    return torch.maximum(q * e, (q - 1) * e).mean()

model = PatchForecaster(n_series=n_series, n_cov=3).cuda()
opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-2)
sched = torch.optim.lr_scheduler.OneCycleLR(opt, 1e-3, total_steps=steps)
model = torch.compile(model)

for y, cov, sid, target in loader:
    with torch.autocast("cuda", dtype=torch.bfloat16):
        loss = pinball(model(y, cov, sid), target, quantiles)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    opt.step(); sched.step(); opt.zero_grad(set_to_none=True)

If you want asymmetric costs, replace the symmetric pinball weights with your real cost ratio — that one-line change is frequently worth more to the business than any architecture upgrade.

Fine-tuning a foundation model: the middle path

You do not have to choose. Fine-tuning Chronos or TimesFM on your own portfolio for a few thousand steps usually beats both zero-shot and a from-scratch model when you have moderate data (hundreds to thousands of series, a couple of years of history). The recipe is ordinary PyTorch: load the checkpoint, low learning rate (1e-4 to 5e-5), short cosine schedule, early stop on a rolling backtest, and freeze nothing unless you are data-starved. Covariates still need to be handled outside the model — commonly by forecasting the residual of a covariate-aware linear or GBDT model, which is an unglamorous but very effective hybrid.

Decision table

Situation	Use
Few series, long clean history, no covariates	ETS / ARIMA, and stop
Tabular features dominate, single short horizon	GBDT on lag + calendar features
Tens of thousands of series, need something this week	Zero-shot Chronos-Bolt / TimesFM
Cold-start or sparse-history series	Zero-shot, or zero-shot blended with seasonal naive
Price, promo, weather plan drive the signal	Trained global model with covariates
Moderate data, want the best accuracy per effort	Fine-tuned foundation model + covariate residual
Asymmetric or cost-weighted errors	Trained model with custom quantile weights
Hierarchy must reconcile	Any base model + reconciliation (MinT / bottom-up)

Shipping it

A forecasting service is mostly plumbing, and the plumbing is where accuracy leaks away:

Version the feature pipeline with the model. A silent change in how a holiday flag is computed is indistinguishable from model decay.
Retrain and re-backtest on a schedule, and keep the backtest report as the artifact you show stakeholders — not a single accuracy number.
Monitor forecast bias, not just error. Persistent one-sided bias is what erodes trust and drives manual overrides.
Track override rate. If planners override 60% of your forecasts, your effective accuracy is theirs, and you should be studying what they know that the model does not.
Export for serving. These models are small; torch.export plus AOTInductor gives you a Python-free artifact that runs a whole portfolio in a nightly batch job on CPU.

The headline of 2026 is that a good forecast no longer requires a training run. The fine print is that beating a good forecast still requires understanding your business's drivers, your cost asymmetry and your hierarchy — and that is where the work has moved. Start zero-shot to establish the floor in a day, then earn every point of improvement with covariates, cost-aware loss and honest backtesting.

If you are standing up a forecasting stack, migrating one off legacy statistical tooling, or trying to work out whether a foundation model is worth it on your data, get in touch — a two-week backtest against your own history usually settles the question.