ArXivCSExplorer
β˜†β˜†BookmarksπŸ†RSSHow to UseFAQ
Built with and by Teycir Ben Soltaneβ€’
How to Useβ€’FAQβ€’GitHubβ€’arXiv.orgβ€’
Share:

20 results for β€œTransformer encoder”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.β“˜

Want pure semantic search? Try claim verification β†’

eess.ASEmpiricalRecentAug 12, 2026

Evaluating Pre-trained Speech Encoders for Spontaneous Speech Detection and Out of Domain Synthetic Speech Generalisation in Indic Languages

Varun Rai, Pavan Kumar J, Sujith Pulikodan, Nihar Desai

This paper evaluates five transformer encoders across 22 Indic languages and conducts a TTS generalisation experiment, revealing an encoder-dependent trade-off between language-discriminability and sp…

View β†’
eess.ASEmpiricalRecentJun 9, 2026

Speech Encoder Fusion for LLM-based Automatic Speech Recognition

Jakob Poncelet, Hugo Van hamme

This paper investigates whether combining multiple pre-trained speech encoders can improve speech-aware large language models for automatic speech recognition.

View β†’
cs.LGcs.CLeess.SPRecentMay 31, 2026

Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding

Athanasios Zeris

The paper introduces Morlet Positional Encoding (MoPE), a novel wavelet-based positional encoding that models position and locality simultaneously, outperforming standard sinusoidal and RoPE methods.

View β†’
cs.CLcs.AIEmpiricalRecentJul 17, 2026

Loop the Loopies!

Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang +3 more

The paper introduces Loopie, two Mixture-of-Experts models that outperform vanilla Transformer baselines in looped Transformers, with extensive ablation studies and a strong reasoning pipeline.

View β†’
cs.SDcs.LGEmpiricalRecentAug 6, 2026

LILAC: An Idempotent Neural Speech Codec

June Young Yi, Dongwook Lee, Jiheum Yeom, Sungroh Yoon

The paper introduces LILAC, a codec idempotent neural audio codec that achieves competitive quality at 0.75 kbit/s and 9.375 Hz.

View β†’
cs.LGcs.AIEmpiricalRecentOct 14, 2026

Your Transformer Is Secretly Elastic

Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts, Cees G. M. Snoek

The paper introduces a retraining-free method to elastify pre-trained transformers into a single model that adapts to varying computational budgets.

View β†’
cs.CLcs.AIRecentMay 28, 2026

Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders

Pierre-Antoine Lequeu, Camille Barboule, Benjamin Piwowarski

The paper proposes explicitly disentangling positional and semantic representations in Transformer encoders, demonstrating that this separation allows for a clearer understanding of how positional inf…

View β†’
cs.ARcs.AIcs.LGEmpiricalRecentJun 15, 2026

NeuronFabric: A Software Reference Architecture for On-Chip Transformer Training with Local Adam

Evgeny Ukladchikov

This paper presents NeuronFabric, a software reference architecture for future FPGA and ASIC implementations of transformer training with local Adam updates.

View β†’
cs.LGcs.AIRecentMay 27, 2026

ReSAE: Residualized Sparse Autoencoders for Multi-Layer Transformer Interventions

Prathyush Poduval, Calvin Yeung, Neel Desai, Mohsen Imani

The paper introduces Residualized Sparse Autoencoders (ReSAEs) to improve multi-layer interventions in transformers by training each layer on the residual activation, which better preserves cross-laye…

View β†’
cs.LGcs.CCRecentJun 1, 2026

Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete

Qian Li, Xinyu Mao, Shang-Hua Teng

The paper demonstrates that positional encodings are not necessary for transformers to achieve universal computation, showing that the inherent mechanism of sliding context windows already provides su…

View β†’
cs.AIEmpiricalRecentOct 16, 2026

FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper

Wei Wang

FUTRANSFORMER is a CUDA implementation of a Llama-style Transformer block for NVIDIA Hopper GPUs

View β†’
cs.CVcs.AIcs.AREmpiricalRecentAug 2, 2026

DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation

Reyhaneh Hosseinzadeh, Parham Zilouchian Moghaddam, Mehdi Modarressi

This paper introduces DeVIT, an acceleration method for vision transformers using differential computation for multiplier-less matrix multiplication and quantization.

View β†’
eess.ASEmpiricalRecentOct 27, 2026

StreamSE: Neural Audio Codec-based Streaming Speech Enhancement for On-device Inference

Hyunjin Choi, Zinu Kim, Junwon Moon, Jeongyun Chae +2 more

The paper proposes StreamSE, a streaming generative speech enhancement model that operates in the latent space of a frozen pretrained neural audio codec.

View β†’
cs.SDcs.AIcs.CREmpiricalRecentJul 18, 2026

Do Speech Tokens Leak Voiceprints? Speaker Inversion Attacks Against End-to-End Speech Language Models

Ye Lu, Yihan Yan, Zhaoyang Zhang, Zhitao Ou +3 more

This paper introduces Audio BERT (AuB) and SpInv, methods for recovering embeddings from speech tokens and performing speaker inversion attacks using only three seconds of frontend output.

View β†’
cs.LGcs.AIcs.AREmpiricalRecentJun 22, 2026

The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model

Mansour Zoubeirou a Mayaki

The paper presents a framework to model and predict energy consumption of Transformer training on multiple GPUs.

View β†’
cs.LGEmpiricalRecentOct 16, 2026

SETAREH Transform: A Sorting-Assisted Haar-Wavelet Sparsification Method for Approximate Inner-Product Computation

Seyed Kamaledin Setarehdan

This paper proposes a Haar-wavelet-based sparsification approach to approximate inner products between high-dimensional vectors.

View β†’