How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation
The paper introduces SearchGEO, a framework for measuring endorsement corruption in large language model-based web-search agents.
The paper introduces a new framework for evaluating the robustness of large language models to attacks.
Before reading this…
Applications
- →Improving the robustness of large language models
To understand this paper, make sure you know these concepts first:
- Knowledge of large language models and web-search agentsfind papers →
Abstract
More Like ThisLarge language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a controlled evaluation framework for measuring endorsement corruption in LLM-based web-search agents, combining a web-evidence manipulation pipeline, a five-mode attack taxonomy, and multiple output-level metrics. We evaluate 13 LLM backends on 308 cases each. Results show that vulnerability patterns vary across backends: overall attack success rate (ASR) ranges from 0.0% on Claude-Sonnet-4.6 to 31.4% on Gemini-3-Flash, the strongest attack mode differs by model family, and the same deployment scaffold could amplify or decrease ASR on different backends. An auxiliary agent-skill probe, where endorsement becomes an install command, exposes a sharp split among otherwise robust backends: Claude over-rejects while GPT over-trusts. These findings argue for treating recommendation reliability under adversarial search content as a first-class dimension of backend safety evaluation.