Time-series language models typically align numerical signals with language using labels, captions, or question-answer pairs.
Although such supervision encourages linguistic representations, it does not explicitly require them to preserve information about how the underlying time-series signal evolves.
We introduce forecast grounding, a pretraining principle that requires language-aligned time-series representations to remain predictive of the future evolution before they are adapted to downstream tasks.
LOTUS Implementation
We instantiate this principle in LOTUS, which divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space.
It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information.
To capture both temporal changes and periodic patterns, we also propose a temporal-spectral patch representation that combines each raw time-series patch with its spectral representation before mapping it into the LLM embedding space.
Performance Results
Across three language model backbones and six downstream evaluation datasets, LOTUS achieves the best average performance with minimal trainable parameters, reaching 59.91% accuracy and outperforming the strongest matched baseline by 3.36%.
Representation Analyses
Our representation analyses further show that LOTUS uses the numerical input and preserves temporal structure in its learned patch representations.
These results establish forecast grounding as a promising pretraining principle, enabling compact time-series language models to outperform models 25 to 50 times their size on TSRBench.