20 results for “Multimodal alignment”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Changhao Xiang, Shangyu Xing, Zhen Wu, Jianbing Zhang +1 more
This paper introduces MultiModal Code-Switching (MMCS), a new pretraining paradigm for multimodal large language models that provides explicit object-level supervision and improves visual grounding an…
Ruizhi Yuan, Zeqiu Yu, Wei Gao, Wei Chen +2 more
This paper proposes a flow-guided target-space alignment method called FlowAlign that improves instance-level correspondence while preserving target-space structure in multimodal alignment problems.
Ilay Kamai, Hugues Van Assel, Aviv Regev, Hagai B. Perets +1 more
This paper develops a unified linear framework to diagnose when cross-modal alignment and prediction succeed or fail in multimodal representation learning.
Hui Wang, Tianyu Ren, Joseph Butler, Christopher Baker +2 more
The paper proposes a framework for multimodal learning under missing modalities, called Latent World Recovery (LWR), which aligns modality-specific embeddings in a shared latent space and fuses availa…
Sarmistha Das, Vaibhav Vishal, Shreyas Guha, Amaan Ali +2 more
This paper introduces a Hybrid Mixture-of-Experts (HybridMoE) framework and a specialized corpus (Varnika) to significantly improve language models' ability to understand and retain figurative, cultur…
Multilingual word-level forced alignment
This paper introduces Align4D, a framework for generating coherent video-3D pairs using any-modal input, achieving state-of-the-art quality and consistency in X-to-4D generation.
The paper introduces RAG-Pref, a novel, training-free Retrieval Augmented Generation (RAG) method for preference alignment that significantly improves LLM refusal guardrails against agentic attacks wi…
Proposes a probabilistic audio-language pretraining framework to model many-to-many correspondence ambiguity in audio-text alignment.
One sentence describing what this paper does
V-LynX is a framework that enhances Video LLMs by integrating new modalities into their existing token interface, achieving state-of-the-art performance across diverse video understanding tasks.
Zixin Zhang, Fan Qi, Shuai Li, Xiaoshan Yang +1 more
The paper proposes FedMChain, a novel federated learning framework that structures multimodal training into sequential phases to mitigate modality competition and improve model performance while reduc…
The authors propose an end-to-end neural architecture for phoneme alignment that outperforms the current state of the art.
The paper introduces MLLM-Microscope, a system that analyzes the internal structure of multimodal large language models (MLLMs), finding that modality fusion significantly impacts the linearity and di…
Tzu-Quan Lin, Chih-Kai Yang, Yi-Cheng Lin, Hao Tang +1 more
This paper investigates how audio and text representations are integrated in large audio-language models.