Soohyeong Shin
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
The paper proposes SISA (SSM-Informed Softmax Attention), a novel hybrid attention mechanism that integrates state-space model (SSM) importance signals directly into the attention score, achieving state-of-the-art performance and superior efficiency.
The paper proposes SISA (SSM-Informed Softmax Attention), a novel hybrid attention mechanism that integrates state-space model (SSM) importance signals directly into the attention score, achieving state-of-the-art performance and superior efficiency.
Papers
Forget Attention: Importance-Aware Attention Is All You Need
The paper proposes SISA (SSM-Informed Softmax Attention), a novel hybrid attention mechanism that integrates state-space model (SSM) importance signals directly into the attention score, achieving sta…