Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently.
We study whether useful memory can also be generated rather than only retrieved.
MemoryAthena's Three Pathways
MemoryAthena uses three pathways: direct Engram retrieval (E), generation from retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH).
Generated memory is conditionally useful: it can complement E in one context but interfere with it in another.
MemoryAthena therefore treats E as an anchor and learns when a generated representation should intervene.
Training and Inference
With the backbone, memory, generators, and readers frozen, a lightweight causal routing head is trained from counterfactual future-token likelihood advantages of GE and GH relative to E.
At inference time, an admitted candidate modifies the E residual through bounded interpolation, while rejection recovers the direct pathway exactly.
Question Answering Performance
On question answering, MemoryAthena raises the five-task average from 37.65 to 39.28 over the direct pathway of the same checkpoint, while the six-task general-NLP average increases from 76.73 to 79.13.
System Parameters and Analyses
The complete memory-side system contains approximately 201M parameters, excluding the frozen backbone.
Further analyses show complementary strengths among E, GE, and GH across tasks and inputs.
Conclusion
These results support generated memory as a selective correction to direct retrieval and highlight routing when, which, and how strongly to intervene as the central challenge.