Learning from Audio-Dependency Errors: Data Curation Strategies Based on Model Confusion Patterns in Audio Question Answering
The authors identify confusion patterns in a large audio-language model and use them to curate diagnostic data for fine-tuning, achieving higher accuracy than the baseline.
The authors propose a method for diagnostic data curation for large audio-language models using model confusion patterns.
Before reading this…
Applications
- →Improving performance of large audio-language models
To understand this paper, make sure you know these concepts first:
- Understanding of natural language processing and machine learning conceptsfind papers →
Abstract
More Like ThisWe frame the system as diagnostic data curation for a large audio-language model: before fine-tuning, we probe Qwen3-Omni-30B-A3B-Instruct under normal, empty-audio, and shuffled-audio conditions to identify how the model's answers change when audio evidence is removed or mismatched. These model confusion patterns are used to bucket training samples into text-prior, shuffle-leak, strong audio-dependent, and hard or misleading cases. Our strongest train-only system fine-tunes only on strong-audio items, where the normal audio-question pair is correct but both counterfactual variants fail, plus a small number of empty-audio negatives and a text-only response normalizer for parse-failed generations. On the official development set, the best train-only system reaches 67.27% accuracy after response normalization, compared with 65.90% for our local Qwen3-Omni baseline. Final submissions additionally include models trained using train+development splits and a three-model ensemble.