CART introduces a parameter-efficient recurrent transformer architecture that reuses a core block multiple times, but its performance does not surpass a dense baseline, suggesting that weight sharing…

The paper introduces Loopie, two Mixture-of-Experts models that outperform vanilla Transformer baselines in looped Transformers, with extensive ablation studies and a strong reasoning pipeline.

The paper analyzes the expressivity of padded transformers, proving that their computational power is primarily determined by model depth and numeric precision, rather than attention type or width.

This paper proposes a hybrid two-stage diffusion transformer architecture for instruction-guided audio editing, balancing performance and efficiency.

This paper isolates the effect of state update design in causal self-attention and introduces structural interventions to reduce approximation errors, outperforming prior post hoc baselines on long-co…

The paper presents an initial implementation of a quantized, integer-only transformer for jet tagging on the AMD Versal AI Engine using a reusable software framework.

The paper analyzes the phase transitions of the noisy transformer model on the unit sphere, proving a sharp global-minimizer dichotomy that depends on the dimension and coupling strength.

This paper proposes KroQuant, a post-training quantization method for diffusion transformers using learned Kronecker-structured invertible transforms, which reduces parameters, improves speed, and mai…

The paper proposes EEG-FuseFormer, a transformer-based framework that fuses features from CNN-LSTM and ResNet-18 to achieve high accuracy in predicting seizure onset from EEG signals.

The paper introduces Morlet Positional Encoding (MoPE), a novel wavelet-based positional encoding that models position and locality simultaneously, outperforming standard sinusoidal and RoPE methods.

The paper demonstrates that content suppression techniques used in language models only mask prohibited content at the output level, failing to eliminate the underlying concepts from the model's inter…