30 papers
Forging More Tasks into Better Harnesses: Agentic Harness Evolution via Successive Scale Racing
Jingzhe Liu, Jiliang Tang, Tong Zhao +2
Agentic harness evolution optimizes an agent's instructions, tools, and control flow by using its task execution experiences as evolving evidence. While more evolution tasks should…
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Liam Collins, Jiwen Ren, Donald Loveland +20
Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling beha…
Preserving Item Semantics for Free: Rethinking Token Initialization in LLM-Based Generative Recommendation
Donald Loveland, Liam Collins, Bhuvesh Kumar +2
Recent advances in generative recommendation (GR) leverage large language models (LLMs) as recommender backbones, enabling LLMs to directly generate recommendations conditioned on…
Item Response Theory for AI Safety
Joshua Fonseca Rivera, Neil Shah, David Demitri Africa +1
Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because b…
Implicit Reasoning for Large Language Model-based Generative Recommendation
Yinhan He, Liam Collins, Bhuvesh Kumar +3
Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge. Yet reliably invoking this kn…
Beyond Fixed Depths and Widths: Optimizing Textual Decoding Tries in LLM-based Generative Recommendation
Jingzhe Liu, Hanbing Wang, Jiliang Tang +4
Generative recommendation (GR) is an increasingly popular paradigm in recommender systems, with a prominent line of work using LLMs as autoregressive backbones to predict the next…
On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies
Sunwoo Kim, Sunkyung Lee, Clark Mingxuan Ju +5
Generative recommendation (GR) has emerged as a promising direction for recommender systems. Recently, large language models (LLMs) have been increasingly adopted for GR, as their…
Consistency Training Along the Transformer Stack
Sukrati Gautam, Neil Shah, Arav Dhoot +7
Consistency training encourages models to behave similarly across different contexts, and has shown promise for reducing misalignment. We broaden the scope of consistency training…
Self-supervised User Profile Generation for Personalization
Clark Mingxuan Ju, Yuwei Qiu, Tong Zhao +1
Personalizing large language models (LLMs) has become a central challenge as LLMs are deployed across recommendation, search, dialogue, and content generation -- settings where the…
Task diversity produces systematic transfer but inhibits continual reinforcement learning
Purab Seth, Neil Shah, Ishaan Sinha +4
Continual reinforcement learning (RL) aims to produce agents that never stop adapting to new tasks. A key question is how this interacts with the diversity of tasks an agent experi…
MLPs are Efficient Distilled Generative Recommenders
Zitian Guo, Yupeng Hou, Clark Mingxuan Ju +2
Generative recommendation models employing Semantic IDs (SIDs) exhibit strong potential, yet their practical deployment is bottlenecked by the high inference latency of beam-expand…
Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation
Yupeng Hou, Haven Kim, Clark Mingxuan Ju +3
Generative recommendation (GR) models generate items by autoregressively producing a sequence of discrete tokens that jointly index the target item. However, this autoregressive ge…
CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
Hansi Zeng, Liam Collins, Bhuvesh Kumar +2
Agentic search -- the task of training agents that iteratively reason, issue queries, and synthesize retrieved information to answer complex questions -- has achieved remarkable pr…
Semantic IDs for Recommender Systems at Snapchat: Use Cases, Technical Challenges, and Design Choices
Clark Mingxuan Ju, Tong Zhao, Leonardo Neves +15
Effective item identifiers (IDs) are an important component for recommender systems (RecSys) in practice, and are commonly adopted in many use cases such as retrieval and ranking.…
FlexRec: Adapting LLM-based Recommenders for Flexible Needs via Reinforcement Learning
Yijun Pan, Weikang Qiu, Qiyao Ma +4
Modern recommender systems must adapt to dynamic, need-specific objectives for diverse recommendation scenarios, yet most traditional recommenders are optimized for a single static…
Exploiting ID-Text Complementarity via Ensembling for Sequential Recommendation
Liam Collins, Bhuvesh Kumar, Clark Mingxuan Ju +4
Modern Sequential Recommendation (SR) models commonly utilize modality features to represent items, motivated in large part by recent advancements in language and vision modeling.…
Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling
Xingyue Huang, Xueying Ding, Mingxuan Ju +3
Softmax attention struggles with long contexts due to structural limitations: the strict sum-to-one constraint forces attention sinks on irrelevant tokens, and probability mass dis…
Masked Diffusion for Generative Recommendation
Kulin Shah, Bhuvesh Kumar, Neil Shah +1
Generative recommendation (GR) with semantic IDs (SIDs) has emerged as a promising alternative to traditional recommendation approaches due to its performance gains, capitalization…
Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
Xueying Ding, Xingyue Huang, Mingxuan Ju +5
Large language models produce powerful text embeddings, but their causal attention mechanism restricts the flow of information from later to earlier tokens, degrading representatio…
Understanding Generative Recommendation with Semantic IDs from a Model-scaling View
Jingzhe Liu, Liam Collins, Jiliang Tang +3
Recent advancements in generative models have allowed the emergence of a promising paradigm for recommender systems (RS), known as Generative Recommendation (GR), which tries to un…
Sequential Data Augmentation for Generative Recommendation
Geon Lee, Bhuvesh Kumar, Clark Mingxuan Ju +4
Generative recommendation plays a crucial role in personalized systems, predicting users' future interactions from their historical behavior sequences. A critical yet underexplored…
Generative Recommendation with Semantic IDs: A Practitioner's Handbook
Clark Mingxuan Ju, Liam Collins, Leonardo Neves +4
Generative recommendation (GR) has gained increasing attention for its promising performance compared to traditional models. A key factor contributing to the success of GR is the s…
A Pre-training Framework for Relational Data with Information-theoretic Principles
Quang Truong, Zhikai Chen, Mingxuan Ju +3
Relational databases underpin critical infrastructure across a wide range of domains, yet the design of generalizable pre-training strategies for learning from relational databases…
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
Ngoc Bui, Menglin Yang, Runjin Chen +5
Backward compatible representation learning enables updated models to integrate seamlessly with existing ones, avoiding to reprocess stored data. Despite recent advances, existing…