LESSRec: Level-Specific History Modeling for Semantic ID-Based Recommendation
This paper proposes a new recommendation model, LESSRec, that improves the effectiveness of semantic ID-based recommendation by restricting attention to historical tokens at the target token's semantic level and providing an item-level semantic sequence.
The proposed LESSRec model improves the effectiveness of semantic ID-based recommendation by restricting attention to historical tokens at the target token's semantic level, which is a novel approach compared to prior work.
Before reading this…
Applications
- →Recommendation systems
- →Personalized recommendation
To understand this paper, make sure you know these concepts first:
- Understanding of recommendation systemsfind papers →
- Knowledge of deep learning modelsfind papers →
Abstract
More Like ThisSemantic ID-based recommendation represents each item as a sequence of semantic tokens obtained through residual quantization. This paradigm generates the next item's tokens autoregressively from coarse to fine, entailing each prediction attending to historical tokens at all semantic levels. However, our inference-time analysis reveals that historical tokens at the target-token's semantic level tend to be more informative, whereas other levels can introduce interference. Motivated by these findings, we propose LESSRec that separately encodes the historical token sequence at each semantic level and restricts attention over historical tokens to the sequence at the target token's semantic level during prediction. To complement these level-specific histories, LESSRec explicitly provides an item-level semantic sequence that represents each historical item's overall semantics in chronological order. The decoder accesses the level-aligned token sequence and the item-level semantic sequence through two separate cross-attention branches and fuses their contexts. We further introduce LESSRec+, which uses separate encoders for different semantic-level histories with the same total parameter count to reduce encoder computation. Extensive experiments across three public datasets and three tokenizers demonstrate the recommendation effectiveness of LESSRec and LESSRec+, with efficiency analyses highlighting the computational benefits of LESSRec+.