9 papers
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Ruizhong Qiu, Yinglong Xia, Dongqi Fu +6
Generative recommendation is an emerging paradigm that has shown promise in industrial recommendation systems, aiming to predict users' next interactions from their historical beha…
DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
Jiarui Feng, Hanqing Zeng, Karish Grover +11
Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performa…
Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback
Weizhi Zhang, Wooseong Yang, Yuxin Cui +9
Traditional recommender systems (RecSys) primarily infer user preferences from implicit signals (such as clicks, watches, and purchases), often neglecting the rich explicit context…
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15
Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRA…
Text Has Curvature
Karish Grover, Hanqing Zeng, Yinglong Xia +2
Does text have an intrinsic curvature? Language is increasingly modeled in curved geometries - hyperbolic spaces for hierarchy, mixed-curvature manifolds for compositional structur…
S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
Hanqing Zeng, Yinglong Xia, Zhuokai Zhao +7
Fine-tuning pre-trained large language models (LLMs) presents a dual challenge of balancing parameter efficiency and model capacity. Existing methods like low-rank adaptations (LoR…