4 papers
Multi-level context Modeling for consistent expert selection in Mixture-of-Experts
Shuhan Huang, Naifan Zhang, Yuanbo Tang +2
Mixture-of-Experts (MoE) enables efficient scaling of Transformer models by routing tokens to a small subset of experts. However, existing routers typically condition expert select…
Data-Efficient and Robust Trajectory Generation through Pathlet Dictionary Learning
Yuanbo Tang, Yan Tang, Zixuan Zhang +2
Trajectory generation has recently drawn growing interest in privacy-preserving urban mobility studies and location-based service applications. Although many studies have used deep…
Dy-mer: An Explainable DNA Sequence Representation Scheme using Dictionary Learning
Zhiyuan Peng, Naifan Zhang, Yuanbo Tang +1
DNA sequences encode critical genetic information, yet their variable length and discrete nature impede direct utilization in deep learning models. Existing DNA representation sche…
Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
Yuanbo Tang, Yan Tang, Naifan Zhang +2
Mixture-of-Experts based large language models (MoE LLMs) have shown significant promise in multitask adaptability by dynamically routing inputs to specialized experts. Despite the…