28 papers
Preserving Item Semantics for Free: Rethinking Token Initialization in LLM-Based Generative Recommendation
Donald Loveland, Liam Collins, Bhuvesh Kumar +2
Recent advances in generative recommendation (GR) leverage large language models (LLMs) as recommender backbones, enabling LLMs to directly generate recommendations conditioned on…
Item Response Theory for AI Safety
Joshua Fonseca Rivera, Neil Shah, David Demitri Africa +1
Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because b…
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Liam Collins, Jiwen Ren, Donald Loveland +19
Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling beha…
Beyond Fixed Depths and Widths: Optimizing Textual Decoding Tries in LLM-based Generative Recommendation
Jingzhe Liu, Hanbing Wang, Jiliang Tang +4
Generative recommendation (GR) is an increasingly popular paradigm in recommender systems, with a prominent line of work using LLMs as autoregressive backbones to predict the next…
Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling
Xingyue Huang, Xueying Ding, Mingxuan Ju +3
Softmax attention struggles with long contexts due to structural limitations: the strict sum-to-one constraint forces attention sinks on irrelevant tokens, and probability mass dis…
On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies
Sunwoo Kim, Sunkyung Lee, Clark Mingxuan Ju +5
Generative recommendation (GR) has emerged as a promising direction for recommender systems. Recently, large language models (LLMs) have been increasingly adopted for GR, as their…