CounterFactual Retrieval for Evidence-Aware Individual Animal Identification
This paper proposes Counterfactual Foreground Retrieval (CFR) to evaluate the performance of animal identification models and provide evidence profiles.
This paper introduces a new evaluation protocol that provides a more comprehensive understanding of animal identification models' performance and behavior.
Before reading this…
Applications
- →Animal identification
- →Computer vision
To understand this paper, make sure you know these concepts first:
- Image retrievalfind papers →
- Animal identification modelsfind papers →
Abstract
More Like ThisAutomated identification of individual animals is moving from small curated studies toward shared benchmarks and reusable foundation models. This progress is useful, but current evaluation leaves the evidence behind a match unresolved, i.e., high retrieval accuracy does not show whether a model used animal features or identity-correlated shortcuts. We address this with Counterfactual Foreground Retrieval (CFR), an evidence-aware evaluation protocol. CFR evaluates the same query-gallery task under different views: full image, foreground-only, and shortcut-only. The full-image measures standard retrieval performance; the foreground-only measures foreground-supported retrieval; and shortcut-only view measure how much identity signal is recoverable from background, or silhouette. This turns a retrieval score into an evidence profile: how well the model performs on the original images, how much of that performance remains when only the animal is visible, and how much identity signal remains recoverable when the animal is removed. We release around 0.5M masks for 34 datasets and code for mask generation and CFR evaluation.