10 citations · 14 across the 6 of their papers we have counts for
6 papers
MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
Shen Yuan, Yin Zheng, Taifeng Wang +2
Adapting large-scale foundation models in multi-task scenarios often suffers from task conflict and oblivion. To mitigate such issues, we propose a novel ''model MoE-ization'' stra…
Towards Better Multi-head Attention via Channel-wise Sample Permutation
Shen Yuan, Hongteng Xu
Transformer plays a central role in many fundamental deep learning models, e.g., the ViT in computer vision and the BERT and GPT in natural language processing, whose effectiveness…
MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Prediction
Fanmeng Wang, Wentao Guo, Minjie Cheng +3
Polymers are high-molecular-weight compounds constructed by the covalent bonding of numerous identical or similar monomers so that their 3D structures are complex yet exhibit unign…
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
Shen Yuan, Haotian Liu, Hongteng Xu
While following different technical routes, both low-rank and orthogonal adaptation techniques can efficiently adapt large-scale pre-training models in specific tasks or domains ba…
Sliceformer: Make Multi-head Attention as Simple as Sorting in Discriminative Tasks
Shen Yuan, Hongteng Xu
As one of the most popular neural network modules, Transformer plays a central role in many fundamental deep learning models, e.g., the ViT in computer vision and the BERT and GPT…
ImDrug: A Benchmark for Deep Imbalanced Learning in AI-aided Drug Discovery
Lanqing Li, Liang Zeng, Ziqi Gao +11
The last decade has witnessed a prosperous development of computational methods and dataset curation for AI-aided drug discovery (AIDD). However, real-world pharmaceutical datasets…