The paper introduces the Triangulated Preference Shift score, an automated, curation-free metric to quantify systematic lexical biases introduced into Large Language Models during the preference-learn…

BayesNCL introduces a probabilistic gating mechanism to resolve the optimization conflict in Contrastive Learning, leading to highly disentangled and semantically consistent representations.

The paper demonstrates that Transformers trained on local comparisons implicitly learn a global, one-dimensional ordinal structure, mirroring the human ability to perform transitive inference.

This paper proposes a unified evaluation framework for hate speech detection that systematically assesses model performance and explainability across various label and rationale representation spaces,…

This paper investigates the use of contrastive objectives for brain decoding using functional MRI (fMRI) activity and shows that linear contrastive decoders outperform other methods.

The paper introduces an Item Response Theory (IRT)-based indicator that effectively identifies likely mislabeled items in existing LLM benchmarks, revealing systematic errors in labeling and model spe…

The paper introduces CERA, a novel contrastive retrieval framework that improves RAG factuality and interpretability by using subjectivity-based hard negative selection and an auxiliary attention alig…

The paper proposes WEINCE, a modified InfoNCE objective that uses extreme value theory corrections to improve contrastive learning by more accurately modeling the selection of hard negative examples.

This paper demonstrates that large language models spontaneously develop geometric structures corresponding to human perceptual domains (like color or pitch) within their internal layers, suggesting t…

This paper explains the concept of 'weight-norm criticality' in deep neural network training, which is a stability issue induced by the interaction between normalization and weight decay.

The paper proposes a utility-aware multimodal contrastive learning framework that optimizes product image generation not just for semantic coherence, but also for maximizing consumer demand in online…

This paper systematically characterises the sensitivity of emergent misalignment (EM) in LLMs to various training choices, finding that the choice of optimiser has the largest effect on misalignment r…

The paper theoretically analyzes the properties that optimal sparse autoencoder (SAE) dictionaries must satisfy, deriving constraints that explain observed SAE behaviors like hierarchical splitting an…

The paper proposes a novel geometric embedding hashing method to recover object correspondences (vector links) between two embedding clouds generated by different black-box encoders using only a small…

The paper introduces COMET, a novel PLS-SVD framework, to analyze the audio-text modality gap in CLAP models, showing that shared concepts are captured by a small subset of axes, and proposes a spectr…

This paper explains how Rotary Position Embeddings (RoPE) frequencies in transformer models correspond to the relative-distance structure of training data, and the implications for long-context genera…

The paper demonstrates that the AI-like style introduced by post-training alignment can be measured, localized, and causally removed using a novel ablation technique called PASTA.

Xetrieval introduces an embedding-level framework to mechanistically explain dense retrieval decisions by decomposing high-dimensional embeddings into sparse, human-interpretable features.