Large Language Models
Research on LLMs, transformers, and language model scaling
20 papers indexed
Artificial Epanorthosis: Why large language models overuse a classical rhetorical figure, and how to mitigate it
This paper measures and analyzes the overuse of epanorthosis, a rhetorical figure, in large language models and proposes techniques to mitigate it.
Attention Is Where You Attack
The paper introduces the Attention Redistribution Attack (ARA), a white-box adversarial method that bypasses safety alignments in LLMs by manipulating the attention mechanism's geometry, showing that…
ARES: Adaptive Red-Teaming and End-to-End Repair of Policy-Reward System
Jiacheng Liang, Yao Ma, Tharindu Kumarage, Satyapriya Krishna +4 more
ARES is a novel framework that systematically discovers and mitigates dual vulnerabilities in RLHF systems by simultaneously testing the core LLM and its Reward Model (RM) using structured adversarial…
Seed Hijacking of LLM Sampling and Quantum Random Number Defense
Ziyang You, Xiaoke Yang, Zhanling Fan, Feng Guo +2 more
The paper introduces SeedHijack, a backdoor attack that manipulates the pseudorandom number generation process in LLMs to force specific token selections, and proposes a hardware quantum random number…
SDR: Set-Distance Rewards for Radiology Report Generation
The paper introduces Set-Distance Rewards (SDR), a permutation-invariant reward signal that effectively guides the generation of unordered radiology reports, significantly outperforming standard train…
UniAE-MoE: A Unified Audio Encoder via Mixture of Experts
Shengbo Cai, Zhisheng Zhang, Zichao Nie, Jing Peng +2 more
This paper introduces UniAE-MoE, a unified audio encoder that achieves state-of-the-art performance in audio understanding tasks.
Target-Side Paraphrase Augmentation for Sign Language Translation with Large Language Models
The paper proposes using GPT-4o to generate controlled paraphrases of target text for sign language translation (SLT) augmentation, achieving significant BLEU-4 improvements on PHOENIX14T.
The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models
This study investigates cross-lingual distributional skew in frontier large language models under adversarial conditions.
Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking
Zekun Qi, Xuchuan Chen, Dairu Liu, Chenghuai Lin +9 more
The paper introduces Humanoid-GPT, a large-scale generative Transformer model that achieves robust zero-shot motion tracking and control by training on a massive, unified corpus of motion data.
TAHOE: Text-to-SQL with Automated Hint Optimization from Experience
The paper presents Tahoe, a system that optimizes Text-to-SQL performance through dynamic data management and hint learning.
In-Context Reward Adaptation for Robust Preference Modeling
The paper proposes In-Context Reward Adaptation, a transformer-based framework that uses in-context learning and auxiliary signals (like human response time) to robustly model diverse and unseen human…
Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows
The paper benchmarks local, offline LLMs for confidential translation workflows, demonstrating that while they are viable for privacy-sensitive use, they generally lag behind top commercial NMT system…
Listwise Cross-Encoder Fine-Tuning vs. Agentic Instruction Tuning for LLM Rerankers: A Systematic Study in Medical Procedure Reranking
This paper compares two reranking paradigms for health insurance information retrieval: small cross-encoders fine-tuned with listwise learning-to-rank objectives and a 4B-parameter instruction reranke…
GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
Zhaoxin Yu, Qi Shen, Hengli Li, Zhaowei Zhang +3 more
The paper introduces GradCuit, a method for optimizing internal reasoning in large language models at test time by inserting optimizable latent states between hidden representations and enabling direc…
Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion
The paper introduces Head-Masked Nullspace Steering (HMNS), a novel geometry-aware attack method that achieves state-of-the-art jailbreak success rates by manipulating the internal attention mechanism…
Gender Bias in LLM Hiring Decisions: Evidence from a Japanese Context and Evaluation of Mitigation Strategies
This study investigates gender bias in Japanese hiring decisions using large language models and proposes practical mitigation strategies.
Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
A multi-agent framework is presented for political coalition formation using LLMs, combining Supervised Fine-Tuning, Direct Preference Optimization, and Retrieval-Augmented Generation.
HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression
Minghui Zheng, Hongxu Chen, Huimin Ren, Hongsheng Xin +7 more
HMPO introduces a single-stage, cost-effective reinforcement learning framework that achieves significant token compression of Chain-of-Thought reasoning with minimal loss of accuracy, applicable acro…
You Are the Trigger: Speaker-Activated Backdoors
This paper introduces a new type of backdoor attack on speech models, called Speaker-Activated Backdoor, where an attacker uses a target speaker's natural voice as the trigger.
Towards Hyperparameter Transfer for Differentially Private Optimization
This paper introduces DP-μP, a framework that extends Maximal Update Parametrization to the differential privacy setting, enabling zero-shot hyperparameter transfer for large-scale models.