148 citations · 168 across the 6 of their papers we have counts for
7 papers
UMoE: Unifying Attention and FFN with Shared Experts
Yuanhang Yang, Chaozheng Wang, Jing Li
Sparse Mixture of Experts (MoE) architectures have emerged as a promising approach for scaling Transformer models. While initial works primarily incorporated MoE into feed-forward…
XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection
Yuanhang Yang, Shiyi Qi, Wenchao Gu +3
Sparse models, including sparse Mixture-of-Experts (MoE) models, have emerged as an effective approach for scaling Transformer models. However, they often suffer from computational…
Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
Bin Hu, Chenyang Zhao, Pu Zhang +4
Large language models (LLMs) encode a vast amount of world knowledge acquired from massive text datasets. Recent studies have demonstrated that LLMs can assist an embodied agent in…
BadCS: A Backdoor Attack Framework for Code search
Shiyi Qi, Yuanhang Yang, Shuzhzeng Gao +2
With the development of deep learning (DL), DL-based code search models have achieved state-of-the-art performance and have been widely used by developers during software developme…
When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods
Zhuo Zhang, Yuanhang Yang, Yong Dai +2
With increasing privacy concerns on data, recent studies have made significant progress using federated learning (FL) on privacy-sensitive natural language processing (NLP) tasks.…
Once is Enough: A Light-Weight Cross-Attention for Fast Sentence Pair Modeling
Yuanhang Yang, Shiyi Qi, Chuanyi Liu +3
Transformer-based models have achieved great success on sentence pair modeling tasks, such as answer selection and natural language inference (NLI). These models generally perform…
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence
Chaozheng Wang, Yuanhang Yang, Cuiyun Gao +3
Pre-trained models have been shown effective in many code intelligence tasks. These models are pre-trained on large-scale unlabeled corpus and then fine-tuned in downstream tasks.…