The paper introduces Loopie, two Mixture-of-Experts models that outperform vanilla Transformer baselines in looped Transformers, with extensive ablation studies and a strong reasoning pipeline.

The paper presents an initial implementation of a quantized, integer-only transformer for jet tagging on the AMD Versal AI Engine using a reusable software framework.

This paper proposes ConsisFormer, a compute-efficient Transformer design for wireless foundation models (WFMs) using short-term channel consistency and adaptive token aggregation.

The paper introduces Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that enables abstract intermediate computation to persist across dec…

The paper introduces Chimera, a highly efficient and scalable MCU designed for ultra-low-power edge AI inference, achieving 3.1 TOPS/W by integrating a dedicated transformer accelerator and a QoS-guar…

Introduces Posterior Prefix Tuning (PPT) for eliciting high-utility continuations from Bayes-filtered transformers using a latent posterior model.

This paper demonstrates that transformer-based policies can provably learn complex tree search mechanisms, such as depth-first search, purely through reinforcement learning in a stochastic environment…

This paper studies the distribution of reinforcement learning (RL) adaptation across transformer layers in large language models and finds that training a single layer can recover most of the gains ob…

This paper proposes a transition-aware backend dispatch approach for efficient large language model inference on edge platforms, reducing latency, energy, and energy-delay product by an average of 17.…

The paper analyzes the expressivity of padded transformers, proving that their computational power is primarily determined by model depth and numeric precision, rather than attention type or width.

The paper introduces Morlet Positional Encoding (MoPE), a novel wavelet-based positional encoding that models position and locality simultaneously, outperforming standard sinusoidal and RoPE methods.

EncFormer is a novel two-party framework that significantly improves the efficiency and scalability of private Transformer inference by optimizing the combination of Fully Homomorphic Encryption (FHE)…

This paper investigates the application of transformer-based diffusion models for simulating and reconstructing hydrological time series using data from six sites in North-East France.

The paper proposes EEG-FuseFormer, a transformer-based framework that fuses features from CNN-LSTM and ResNet-18 to achieve high accuracy in predicting seizure onset from EEG signals.