Felipe Urrutia
1 indexed paper
Recent (6 mo)
1With code
0Influential cites
0Benchmarked
0Publications per year
126
Top categories
ML×1AI×1
Frequent co-authors
Research Timeline
2026
Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization
The paper analyzes the distinct computational roles of positional versus symbolic attention heads in Transformers, demonstrating that symbolic mechanisms generalize more reliably to longer sequences than positional ones.
Highlighted terms show continued research focus across papers