8 papers
XPERT: Expert Knowledge Transfer for Effective Training of Language Models
Chang Liu, Boyu Shi, Xu Yang +1
Mixture-of-Experts (MoE) language models organize knowledge into explicitly routed expert modules, making expert-level representations traceable and analyzable. By analyzing expert…
DCGL: Dual-Channel Graph Learning with Large Language Models for Knowledge-Aware Recommendation
Xinchi Zou, Tongzhenzhi Su, Jianjun Li +4
Knowledge Graphs (KGs) have proven highly effective for recommendation systems by capturing latent item relationships, while recent integration of Large Language Models (LLMs) has…
Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
Boyu Shi, Chang Liu, ChuanBao Gao +2
Layer pruning efficiently reduces Large Language Model (LLM) computational costs but often triggers sudden performance collapse. Existing representation-based analyses struggle to…
Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
Jinqian Chen, Chang Liu, Jihua Zhu
Federated LoRA enables parameter-efficient adaptation of large language models under decentralized data and limited client resources.However, directly averaging LoRA factors is rep…
Learngene Search Across Multiple Datasets for Building Variable-Sized Models
Boyu Shi, Junbo Zhou, Chang Liu +3
Deep learning methods are widely used under diverse resource constraints, resulting in models of varying sizes, such as the Vision Transformer (ViT) series. Deploying these models…
ShapleyPipe: Hierarchical Shapley Search for Data Preparation Pipeline Construction
Jing Chang, Chang Liu, Jinbin Huang +3
Automated data preparation pipeline construction is critical for machine learning success, yet existing methods suffer from two fundamental limitations: they treat pipeline constru…