This paper evaluates the performance of a Large Language Model (LLM) in a high-stakes context by comparing it to human experts and measuring variance and error magnitude.

This paper explores the effect of conversational timing properties on automatic speech recognition (ASR) systems by controlling and optimizing pause and overlap timing distributions.

This paper investigates how the final prompt in conversational AI-search evaluations differs from the conversation history, using two corpora of commercial and PRISM conversations.

The paper introduces Dialogs, a new Russian conversational speech corpus with high-quality recordings, segmented utterances, and expressive prosody labels.

This paper proposes a scalable framework for multi-domain dialogue state tracking using pretrained BERT model for zero-shot generalization.

The paper introduces MIDI, a novel multilingual dataset that embeds idioms in realistic sentence and conversational contexts across diverse resource levels, revealing that idiom comprehension is signi…

The paper introduces RHELM, a new benchmark designed to test LLMs' long-term memory by simulating realistic, complex, and evolving dialogues that integrate multiple heterogeneous data sources.

The paper introduces the Real World Voice EQ Bench, a multidimensional benchmark for evaluating voice AI across text-to-speech, speech-to-speech, speech understanding, and automatic speech recognition…

The paper introduces FRANZ, a communicative audit framework, to evaluate how LLMs frame responses to subjective questions, finding that LLMs exhibit statistically significant and coupled differences i…

The paper introduces SPEARBench, a benchmark for evaluating naturalness in speech-to-speech language models using a multidimensional protocol.

This paper introduces model merging as a training-free strategy for designing a single retrieval model that operates across both ad-hoc and conversational settings, improving ad-hoc search capabilitie…

The paper introduces BEA-Dialogue+, an expanded 200-hour corpus for Hungarian conversational ASR, demonstrating that while larger data is challenging, specialized fine-tuning techniques significantly…

This paper investigates the effectiveness of stage-dependent preference elicitation strategies in conversational recommendation systems and introduces COPE, a novel architecture for strategy modeling.

CA-BED is a novel framework that improves LLM performance in interactive question-answering by integrating Bayesian Experimental Design to strategically select questions that maximize information gain…

Analyzing longitudinal data from 12,000 Copilot users, the paper finds that individual user habits regarding LLM interaction are highly sticky and difficult to change, and that existing datasets may o…

The paper introduces DialogPII, a multilingual dataset for personal information detection in conversational data, with dialogs generated semi-automatically, manually curated, and transcribed for align…

The paper introduces RUMBA, a new benchmark for long-term conversational memory in LLMs with a fine-grained taxonomy and unified methodology.

This paper proposes a multi-turn retrieval-augmented generation pipeline for conversational systems across four domains.

The paper introduces RefMem-Bench, a new benchmark for measuring reflective memory in long-horizon dialogue, and proposes REMIND, a framework that significantly improves models' ability to synthesize…

The authors identify confusion patterns in a large audio-language model and use them to curate diagnostic data for fine-tuning, achieving higher accuracy than the baseline.