This paper argues that large language models have fundamental limitations as general-purpose solvers due to the capacity-limited nature of language as a communication channel and alignment constraints…

This paper compares the preservation of linguistic content in different text encodings (tokens, bytes, pixels) using a shared bottleneck, revealing their distinct strengths in surface form preservatio…

This paper develops a theory for speculative decoding in practical language model inference systems, where success is governed by local ranking and threshold events.

The paper analyzes language generation and identification in the limit under bounded memory, showing that memory constraints significantly alter learnability, particularly affecting achievable density…

The paper establishes information-theoretic lower bounds for stochastic optimization using low-bit gradients by reducing the problem to compressed Gaussian mean estimation, yielding sharp bounds on co…

The paper analyzes order-agnostic language models (OALMs), finding that their learned conditionals are not true factorizations and proposing a variance-based diagnostic to compare the quality of diffe…

This paper derives bounds on the probability of incorrect clustering of noisy short sequences using statistically optimal rules, focusing on DNA storage decoders.

A new boosting algorithm that strong learns concept classes closed under O(log 1/γ)-XOR using O(log 1/ε) calls to a γ-advantage weak learner and additional samples, by connecting boosting with list-de…

The paper introduces PACZero, a novel PAC-private fine-tuning mechanism that achieves usable utility for large language models while providing strong resistance against membership-inference attacks.

The paper enhances French parsing accuracy by integrating data from a syntactic lexicon and applying word clustering methods to verbs within a Probabilistic Context-Free Grammar framework.

This paper investigates the relationship between language model perplexity and word error rate in modern automatic speech recognition systems, studying the impact of external language models, encoder…

The paper proposes $D^3$, a dynamic graph-constrained scheduling framework that optimizes LLM training order by modeling sample interactions as a dynamic influence graph.

This paper introduces a new scaling law for sparse language models trained with limited data, demonstrating that sparsity can significantly improve performance and delay data saturation during multi-e…

The paper proposes a graph-constrained approach to scale multi-hop training data by decoupling path discovery from path verbalization, significantly expanding the usable corpus size for LLMs.

This paper investigates whether model compression techniques (like quantization and pruning) preserve a Large Language Model's ability to quantify its own uncertainty, finding that accuracy-only evalu…