首页 > AI前沿 > A Transformer-Derived Iterative Preconditioner

A Transformer-Derived Iterative Preconditioner

arXiv机器学习 2026-11-17 23:35 8 阅读 查看原文

Transformers trained on in-context prediction problems can learn known numerical algorithms, e.g., gradient descent. We go further, showing that their internal computations can reveal new algorithms with provable guarantees. A softmax-attention transformer trained to solve linear systems $Xw=y$ implicitly improves matrix conditioning