Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation
This paper investigates whether large language models can faithfully reproduce an individual's behavior and decision logic over time in a paper-trading study.
This paper highlights the gap between coarse behavioral agreement and individual-level fidelity in user simulation, which is a novel contribution to the field.
Keywords
Before reading this…
Applications
- →Improving user simulation models
- →Developing more accurate decision-making models
To understand this paper, make sure you know these concepts first:
- Understanding of large language models and user simulationfind papers →
- Knowledge of paper-trading studiesfind papers →
Abstract
More Like ThisLarge language models (LLMs) are increasingly used as user simulators, but can they faithfully reproduce an individual's behavior and decision logic over time? We investigate this question in a longitudinal paper-trading study with 80 participants. Using rolling next-day predictions, we evaluate trading activity, buy-sell structure, asset selection, and portfolio outcomes, complemented by controlled input ablations and participant-rated explanation alignment. No evaluated LLM reliably outperforms a simple recent-activity persistence baseline for predicting trade occurrence, while buy-sell structure and asset selection remain difficult to reproduce. Similar activity-level performance can also yield substantially different portfolio trajectories. Ablations show that activity predictions depend primarily on recent trades, whereas asset choices are sensitive to multiple evidence sources. Participants rate explanations as more aligned with their recurring strategies than with their decision-specific rationales. These findings reveal a gap between coarse behavioral agreement and individual-level fidelity, motivating separate evaluation of behavioral accuracy, explanation alignment, and evidence dependence in personalized user simulation.