13 papers
Decision-Oriented Recommendation Reranking: An Empirical Study of Its Promise and Limitations
Hanjia Lyu, Yinglong Xia
Large language models (LLMs) have shown promise for recommendation reranking, but their use introduces an important trade-off between recommendation quality and serving efficiency.…
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Jacqueline He, Howard Yen, Shuyue Stella Li +9
Logit-based knowledge distillation (KD) is used to train smaller language models (LMs) via supervision from stronger teachers, but whether its benefits are consistent across traini…
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Ruizhong Qiu, Yinglong Xia, Dongqi Fu +6
Generative recommendation is an emerging paradigm that has shown promise in industrial recommendation systems, aiming to predict users' next interactions from their historical beha…
DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
Jiarui Feng, Hanqing Zeng, Karish Grover +11
Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performa…
Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback
Weizhi Zhang, Wooseong Yang, Yuxin Cui +9
Traditional recommender systems (RecSys) primarily infer user preferences from implicit signals (such as clicks, watches, and purchases), often neglecting the rich explicit context…
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15
Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRA…
Text Has Curvature
Karish Grover, Hanqing Zeng, Yinglong Xia +2
Does natural language text have an intrinsic curvature? Language is increasingly modeled in curved geometries - hyperbolic spaces for hierarchy, mixed-curvature manifolds for compo…
Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
Gangda Deng, Yuxin Yang, Ömer Faruk Akgül +4
Graph Neural Networks (GNNs) have become essential tools for learning on relational data, yet the performance of a single GNN is often limited by the heterogeneity present in real-…
Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
Mufei Li, Dongqi Fu, Limei Wang +10
Modern long-context large language models (LLMs) perform well on synthetic "needle-in-a-haystack" (NIAH) benchmarks, but such tests overlook how noisy contexts arise from biased re…
S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
Hanqing Zeng, Yinglong Xia, Zhuokai Zhao +7
Fine-tuning pre-trained large language models (LLMs) presents a dual challenge of balancing parameter efficiency and model capacity. Existing methods like low-rank adaptations (LoR…
Retrieval Augmentation via User Interest Clustering
Hanjia Lyu, Hanqing Zeng, Yinglong Xia +2
Many existing industrial recommender systems are sensitive to the patterns of user-item engagement. Light users, who interact less frequently, correspond to a data sparsity problem…
TASER: Temporal Adaptive Sampling for Fast and Accurate Dynamic Graph Representation Learning
Gangda Deng, Hongkuan Zhou, Hanqing Zeng +5
Recently, Temporal Graph Neural Networks (TGNNs) have demonstrated state-of-the-art performance in various high-impact applications, including fraud detection and content recommend…
Mixture of Weak & Strong Experts on Graphs
Hanqing Zeng, Hanjia Lyu, Diyi Hu +2
Realistic graphs contain both (1) rich self-features of nodes and (2) informative structures of neighborhoods, jointly handled by a Graph Neural Network (GNN) in the typical setup.…