The paper introduces and analyzes several novel data appraisal metrics, including the Vendi Score and matrix spectral functions, demonstrating that efficient optimization techniques make these metrics…

This paper derives deterministic limits for transfer learning performance of linear discriminant analysis in high-dimensional two-class classification under spiked covariance models.

This paper proposes a method for quantizing large language models with cross-layer error compensation and finite-sample feature-statistics matching.

The scaling exponent in neural scaling laws is not fixed but systematically depends on the optimizer used, with preconditioned optimizers generally yielding steeper scaling.

This paper introduces survey sampling techniques to estimate or minimize empirical pairwise loss functions, showing that targeting informative pairs significantly reduces computational cost while main…

This paper establishes a connection between neural network approximation of score functions and approximation of probability distributions generated by reverse diffusion models.

The paper introduces CalArena, a large-scale, standardized benchmark covering nearly 2000 experiments to comprehensively evaluate post-hoc calibration methods, finding that smooth calibration function…

The paper proposes an extended version of Hypencoder, a retrieval approach that encodes queries as shallow neural networks, achieving comparable effectiveness with fewer active parameters and higher s…

This paper explores the convergence of AdamW optimizer under heavy-tailed assumptions in large language models and proposes a corridor lower-bound mechanism.

The paper introduces Entropic Projection Alignment (EPA), a unified framework that estimates, explains, and improves model performance under distribution shift by aligning source and target distributi…

This paper proposes a two-stage clustering algorithm to ensure per-sample guardrails in LLM-based applications, reducing inference cost and latency.

This paper identifies prediction bias, a failure mode of entropy minimization in test-time adaptation, and proposes Distribution Shift Bias Reduction (DSBR) to stabilize adaptation and prevent model c…

The paper introduces Score Broadcast and Decorrelation (SBD), a general theoretical framework that unifies broadcast-based credit assignment across various differentiable loss functions by leveraging…

A new boosting algorithm that strong learns concept classes closed under O(log 1/γ)-XOR using O(log 1/ε) calls to a γ-advantage weak learner and additional samples, by connecting boosting with list-de…

The paper proposes WARDEN, a distributionally robust adversarial training framework that significantly reduces LLM vulnerability to adversarial attacks by dynamically reweighting hard adversarial exam…

The paper proposes a novel online learning algorithm that achieves an interval regret bound scaling with gradient variation, providing strong theoretical guarantees for non-stationary environments.

This paper analyzes the poor performance of Meta-learning for Training-data Selection (MTS) and proposes that increasing the batch size and incorporating informative features can significantly improve…

The paper proposes a novel, theoretically-grounded algorithm (HAMU) that addresses the challenge of machine unlearning by guaranteeing specified improvements in forget quality while minimizing retain…

The paper introduces trust functions to filter weak supervision labels, enabling near-lossless weak-to-strong generalization by selectively training a strong student using only the most reliable weak…