The paper argues that large activation spikes in LLMs are structural vector biases, and proposes a novel quantization framework (INSERTQUANT) to eliminate these spikes, enabling robust low-bit quantiz…

This paper reviews various mechanisms for calculating word embeddings, investigates popular toolkits and matrices, and experiments with selected implementations.

The paper introduces an efficient, novel algorithm for incremental Byte Pair Encoding (BPE) tokenization that processes input text prefix by prefix, achieving significant speedups and enabling streami…

The paper proposes lightweight fine-tuning as an alternative to pooling-aware training for reducing vector count in late interaction models, achieving broad gains and high compression rates.

The paper introduces Fast-TurboQuant, a multiplier-free projection architecture for large language models that uses a structured fast Johnson-Lindenstrauss transform instead of dense matrices, resulti…

This paper introduces GEAR, a method for training a vector-quantized tokenizer and an autoregressive generator jointly and end-to-end, resolving the issue of non-differentiable VQ indices.

Clark Hash is a stateless, deterministic quantization method that significantly reduces the storage size of neural embeddings while maintaining high accuracy for cosine similarity search.

The paper introduces VibeVoice-ASR-BitNet, a compressed real-time speech recognition model optimized for edge CPUs using heterogeneous quantization and custom SIMD kernels.

PARCEL introduces a novel visual tokenization architecture that combines spatial pooling anchors with conditioned elastic queries, efficiently reducing the computational cost of large Vision-Language…

LoSATok proposes a low-dimensional semantic-acoustic tokenizer that efficiently compresses high-dimensional audio features into a compact latent space, significantly improving the performance and effi…

This paper systematically analyzes combining dimensionality reduction and quantization to compress text embeddings, showing that this combined approach achieves substantial compression (e.g., 0.1% siz…

The paper introduces CFGzip, an offline token space compression technique that significantly reduces the computational overhead of constrained decoding, making complex grammar enforcement feasible at…

This paper compares the preservation of linguistic content in different text encodings (tokens, bytes, pixels) using a shared bottleneck, revealing their distinct strengths in surface form preservatio…

The paper proposes a unified framework for designing efficient and expressive token mixing layers by separating the direct and recurrent influences of inputs, allowing for a principled trade-off betwe…

This study benchmarks token-optimized formats (TOON and TRON) against JSON in end-to-end agentic AI systems, finding that TRON significantly reduces token overhead with minimal performance degradation…

InfoMerge is a novel, training-free method that significantly compresses visual tokens for Video-LLMs by estimating temporal redundancy and allocating tokens based on content richness, achieving high…

This paper introduces Byte-Prefix Marginalization (BPM), a method for consolidating open-weight language models through on-policy distillation while preserving teacher probability mass and maintaining…

The paper introduces Single-stage Sparse Retrieval (SSR), a method that replaces computationally expensive vector clustering with sparse autoencoding to achieve highly efficient multi-vector retrieval…