Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models
This paper introduces explicit personality conditioning and evaluation framework for Multimodal Large Language Models.
This paper introduces a new method for personality conditioning and evaluation in Multimodal Large Language Models, which is not addressed in prior work.
Keywords
Before reading this…
Applications
- →Image captioning
- →Visual question answering
To understand this paper, make sure you know these concepts first:
- Multimodal Large Language Modelsfind papers →
- Personality conditioningfind papers →
Abstract
More Like ThisWith the widespread deployment of Multimodal Large Language Models (MLLMs) in social interaction, understanding and controlling their behavior under complex personality conditions is essential. This paper introduces explicit personality conditioning and establishes a systematic evaluation framework encompassing single-personality induction, multi-personality induction, and personality switching. Experiments show that personality induction improves image captioning performance but can impair performance on tasks requiring precise reasoning, such as visual question answering (VQA). Balancing and residual effects are observed during multi-trait composition and dynamic switching, indicating that model behavior is co-modulated by both previous and current personality constraints. Existing prompt-based personality induction methods show limited transferability to multimodal settings. Our work reveals the dynamic and complex nature of personality modeling in MLLMs and underscores the need for robust, tailored methods for personality induction and evaluation. The code will be released when the paper is accepted.