7 papers · 1 filter
Decision-Oriented Recommendation Reranking: An Empirical Study of Its Promise and Limitations
Hanjia Lyu, Yinglong Xia
Large language models (LLMs) have shown promise for recommendation reranking, but their use introduces an important trade-off between recommendation quality and serving efficiency.…
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Jacqueline He, Howard Yen, Shuyue Stella Li +9
Logit-based knowledge distillation (KD) is used to train smaller language models (LMs) via supervision from stronger teachers, but whether its benefits are consistent across traini…
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Ruizhong Qiu, Yinglong Xia, Dongqi Fu +6
Generative recommendation is an emerging paradigm that has shown promise in industrial recommendation systems, aiming to predict users' next interactions from their historical beha…
DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
Jiarui Feng, Hanqing Zeng, Karish Grover +11
Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performa…
Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback
Weizhi Zhang, Wooseong Yang, Yuxin Cui +9
Traditional recommender systems (RecSys) primarily infer user preferences from implicit signals (such as clicks, watches, and purchases), often neglecting the rich explicit context…
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15
Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRA…
Text Has Curvature
Karish Grover, Hanqing Zeng, Yinglong Xia +2
Does natural language text have an intrinsic curvature? Language is increasingly modeled in curved geometries - hyperbolic spaces for hierarchy, mixed-curvature manifolds for compo…