The Topological Trouble With Transformers
TL;DR
A perspective on why feedforward transformers struggle with dynamic state tracking, and how recurrence may better support temporally extended cognition.
Venue
arXiv
BibTeX
@article{mozer2026topological,
title={The Topological Trouble With Transformers},
author={Michael C. Mozer and Shoaib Ahmed Siddiqui and Rosanne Liu},
year={2026},
eprint={2604.17121},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
title={The Topological Trouble With Transformers},
author={Michael C. Mozer and Shoaib Ahmed Siddiqui and Rosanne Liu},
year={2026},
eprint={2604.17121},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Date
April, 2026
Links