6 citations · 10 across the 21 of their papers we have counts for
5 papers · 1 filter
MH-MoE: Multi-Head Mixture-of-Experts
Shaohan Huang, Xun Wu, Shuming Ma +1
Multi-Head Mixture-of-Experts (MH-MoE) demonstrates superior performance by using the multi-head mechanism to collectively attend to information from various representation spaces…
Textual Aesthetics in Large Language Models
Lingjie Jiang, Shaohan Huang, Xun Wu +1
Image aesthetics is a crucial metric in the field of image generation. However, textual aesthetics has not been sufficiently explored. With the widespread application of large lang…
Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation
Xun Wu, Shaohan Huang, Furu Wei
Recent studies have demonstrated the exceptional potentials of leveraging human preference datasets to refine text-to-image generative models, enhancing the alignment between gener…
Multi-Head Mixture-of-Experts
Xun Wu, Shaohan Huang, Wenhui Wang +1
Sparse Mixtures of Experts (SMoE) scales model capacity without significant increases in training and inference costs, but exhibits the following two issues: (1) Low expert activat…
Mixture of LoRA Experts
Xun Wu, Shaohan Huang, Furu Wei
LoRA has gained widespread acceptance in the fine-tuning of large pre-trained models to cater to a diverse array of downstream tasks, showcasing notable effectiveness and efficienc…