Beyond Patches: Superpixel Token-based Transformers for Attribute-Specific Fashion Retrieval
This paper proposes SuperFashion, a new framework for attribute-specific fashion retrieval that uses superpixel tokens within a Transformer architecture to improve fine-grained image retrieval.
SuperFashion is the first framework to use superpixel tokens within a Transformer architecture for attribute-specific fashion retrieval.
Before reading this…
Applications
- →Web-based image retrieval
To understand this paper, make sure you know these concepts first:
- Transformer architecturefind papers →
- Attention mechanismsfind papers →
Abstract
More Like ThisAttribute-Specific Fashion Retrieval (ASFR) aims to improve fine-grained image retrieval by focusing on specific attributes. However, existing patch-based attention and Transformer methods often misalign with irregular attribute regions and are prone to background noise, limiting their ability to capture subtle, pixel-level microstructures. To tackle these challenges, we propose SuperFashion, the first ASFR framework that adopts superpixel tokens within a Transformer architecture. SuperFashion initially employs an attribute-guided attention mechanism to extract attribute-related features, which in turn guide the cropping of semantically meaningful image regions. Superpixel segmentation is then leveraged on these regions to generate compact, semantically coherent superpixel tokens. By incorporating modality-specific embeddings for both attribute and superpixel tokens, the superpixel token-based Transformer facilitates adaptive interaction and fusion, thereby enhancing attribute localization and discrimination. Extensive experiments on FashionAI, DARN, and DeepFashion demonstrate relative overall MAP improvements of 1.84%, 9.27%, and 9.35% over prior SOTA. SuperFashion offers a new solution for web-based image retrieval.