20 results for βTransformer encoderβ
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.β
Want pure semantic search? Try claim verification β
This paper evaluates five transformer encoders across 22 Indic languages and conducts a TTS generalisation experiment, revealing an encoder-dependent trade-off between language-discriminability and spβ¦
This paper investigates whether combining multiple pre-trained speech encoders can improve speech-aware large language models for automatic speech recognition.
The paper introduces Morlet Positional Encoding (MoPE), a novel wavelet-based positional encoding that models position and locality simultaneously, outperforming standard sinusoidal and RoPE methods.
Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang +3 more
The paper introduces Loopie, two Mixture-of-Experts models that outperform vanilla Transformer baselines in looped Transformers, with extensive ablation studies and a strong reasoning pipeline.
The paper introduces LILAC, a codec idempotent neural audio codec that achieves competitive quality at 0.75 kbit/s and 9.375 Hz.
The paper introduces a retraining-free method to elastify pre-trained transformers into a single model that adapts to varying computational budgets.
The paper proposes explicitly disentangling positional and semantic representations in Transformer encoders, demonstrating that this separation allows for a clearer understanding of how positional infβ¦
This paper presents NeuronFabric, a software reference architecture for future FPGA and ASIC implementations of transformer training with local Adam updates.
The paper introduces Residualized Sparse Autoencoders (ReSAEs) to improve multi-layer interventions in transformers by training each layer on the residual activation, which better preserves cross-layeβ¦
The paper demonstrates that positional encodings are not necessary for transformers to achieve universal computation, showing that the inherent mechanism of sliding context windows already provides suβ¦
FUTRANSFORMER is a CUDA implementation of a Llama-style Transformer block for NVIDIA Hopper GPUs
This paper introduces DeVIT, an acceleration method for vision transformers using differential computation for multiplier-less matrix multiplication and quantization.
Hyunjin Choi, Zinu Kim, Junwon Moon, Jeongyun Chae +2 more
The paper proposes StreamSE, a streaming generative speech enhancement model that operates in the latent space of a frozen pretrained neural audio codec.
Ye Lu, Yihan Yan, Zhaoyang Zhang, Zhitao Ou +3 more
This paper introduces Audio BERT (AuB) and SpInv, methods for recovering embeddings from speech tokens and performing speaker inversion attacks using only three seconds of frontend output.
The paper presents a framework to model and predict energy consumption of Transformer training on multiple GPUs.
This paper proposes a Haar-wavelet-based sparsification approach to approximate inner products between high-dimensional vectors.