10 citations · 14 across the 6 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Towards Better Multi-head Attention via Channel-wise Sample Permutation
Shen Yuan, Hongteng Xu
Transformer plays a central role in many fundamental deep learning models, e.g., the ViT in computer vision and the BERT and GPT in natural language processing, whose effectiveness…
cs.LG2024★ 3 cited
MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Prediction
Fanmeng Wang, Wentao Guo, Minjie Cheng +3
Polymers are high-molecular-weight compounds constructed by the covalent bonding of numerous identical or similar monomers so that their 3D structures are complex yet exhibit unign…
cs.LG2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
Shen Yuan, Haotian Liu, Hongteng Xu
While following different technical routes, both low-rank and orthogonal adaptation techniques can efficiently adapt large-scale pre-training models in specific tasks or domains ba…