Existing sequence models share a critical limitation: they lack an explicit, interpretable mechanism for content-aware temporal recall.
LSTMs compress everything into a single hidden state, thereby losing information about specific past events.
Transformers attend to all tokens indiscriminately at quadratic cost, while state-space models like Mamba compress history into a fixed-dimensional state without explicit content-addressable memory.
To address this gap, we introduce a novel framework grounded in nonlinear time-delay systems, in which the effective delay is dynamically estimated based on the semantic similarity between current and historical inputs.
Our approach features three key innovations:
-
(i) a differentiable semantic lag estimator that computes content-aware soft attention weights over history, producing a weighted historical context and similarity score;
-
(ii) a diagonal state space with closed-form cumulative sum solution, achieving O(1) per-step complexity with no sequential loops;
-
(iii) an interpretable fuzzy semantic modulation mechanism that learns context-dependent rules for balancing instantaneous processing versus historical recall, where rule centers correspond to prototypical semantic contexts.
Experimental results on text classification and generation outperform state-of-the-art methods.