85 citations · 96 across the 15 of their papers we have counts for
10 papers · 1 filter
Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection
Ziqing Fan, Siyuan Du, Shengchao Hu +5
Selecting high-quality pre-training data for large language models (LLMs) is crucial for enhancing their overall performance under limited computation budget, improving both traini…
Continual Task Learning through Adaptive Policy Self-Composition
Shengchao Hu, Yuhang Zhou, Ziqing Fan +4
Training a generalizable agent to continually learn a sequence of tasks from offline trajectories is a natural requirement for long-lived agents, yet remains a significant challeng…
Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
Ziqing Fan, Shengchao Hu, Yuhang Zhou +4
The purpose of offline multi-task reinforcement learning (MTRL) is to develop a unified policy applicable to diverse tasks without the need for online environmental interaction. Re…
Reconstruct the Pruned Model without Any Retraining
Pingjie Wang, Ziqing Fan, Shengchao Hu +3
Structured pruning is a promising hardware-friendly compression technique for large language models (LLMs), which is expected to be retraining-free to avoid the enormous retraining…
Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping
Ziqing Fan, Jiangchao Yao, Ruipeng Zhang +3
Statistical heterogeneity severely limits the performance of federated learning (FL), motivating several explorations e.g., FedProx, MOON and FedDyn, to alleviate this problem. Des…
Federated Learning with Bilateral Curation for Partially Class-Disjoint Data
Ziqing Fan, Ruipeng Zhang, Jiangchao Yao +3
Partially class-disjoint data (PCDD), a common yet under-explored data formation where each client contributes a part of classes (instead of all classes) of samples, severely chall…
Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization
Ziqing Fan, Shengchao Hu, Jiangchao Yao +4
In federated learning (FL), the multi-step update and data heterogeneity among clients often lead to a loss landscape with sharper minima, degenerating the performance of the resul…
HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
Shengchao Hu, Ziqing Fan, Li Shen +3
The purpose of offline multi-task reinforcement learning (MTRL) is to develop a unified policy applicable to diverse tasks without the need for online environmental interaction. Re…
Q-value Regularized Transformer for Offline Reinforcement Learning
Shengchao Hu, Ziqing Fan, Chaoqin Huang +4
Recent advancements in offline reinforcement learning (RL) have underscored the capabilities of Conditional Sequence Modeling (CSM), a paradigm that learns the action distribution…
FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation
Ziqing Fan, Yanfeng Wang, Jiangchao Yao +3
The statistical heterogeneity of the non-independent and identically distributed (non-IID) data in local clients significantly limits the performance of federated learning. Previou…