Time Series Foundation Models (TSFMs) promise a paradigm shift toward zero-shot forecasting by eliminating task-specific training.
However, existing works often overlook trade-offs between predictive accuracy and probabilistic calibration.
This paper presents a benchmark study of six TSFMs evaluated on energy, traffic, and financial datasets.
We contrast their performance against statistical baselines and a supervised DL model.
The study reveals that while TSFMs outperform statistical methods and supervised models, they are subject to a fundamental trade-off between point accuracy and probabilistic reliability.
Specifically, xLSTM architectures provide robust probabilistic calibration across horizons.
In contrast, patch-based transformers offer competitive accuracy but face calibration issues at long horizons, while transformer-based models exhibit context saturation points for optimal zero-shot reasoning.
These findings offer evidence-based guidance for balancing generalization and uncertainty quantification in real-world deployments.