72 citations · 81 across the 10 of their papers we have counts for
10 papers
Correlation-Aware Structured Pruning for Large Language Models
Sicheng Xu, Hao Shi, Wei Zhang +6
Structured pruning is a promising approach for reducing the substantial inference costs of Large Language Models (LLMs) while maintaining hardware efficiency. Many existing methods…
SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs
Chao Pan, Yu Wu, Xin Yao
Internal Safety Collapse (ISC) is a failure mode in which frontier LLMs, when executing legitimate professional tasks whose correct completion structurally requires harmful content…
Scalable Training of Mixture-of-Experts Models with Megatron Core
Zijie Yan, Hongxiao Bai, Xin Yao +42
Scaling Mixture-of-Experts (MoE) training introduces systems challenges absent in dense models. Because each token activates only a subset of experts, this sparsity allows total pa…
Evaluating Large Language Models on Multimodal Chemistry Olympiad Exams
Yiming Cui, Xin Yao, Yuxuan Qin +3
Multimodal scientific reasoning remains a significant challenge for large language models (LLMs), particularly in chemistry, where problem-solving relies on symbolic diagrams, mole…
ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding
Xingqi Wang, Yiming Cui, Xin Yao +3
Large Vision-Language Models (LVLMs) have recently demonstrated remarkable progress, yet hallucination remains a critical barrier, particularly in chart understanding, which requir…
Mozart's Touch: A Lightweight Multi-modal Music Generation Framework Based on Pre-Trained Large Models
Jiajun Li, Tianze Xu, Xuesong Chen +2
In recent years, AI-Generated Content (AIGC) has witnessed rapid advancements, facilitating the creation of music, images, and other artistic forms across a wide range of industrie…
Rethinking LLM Language Adaptation: A Case Study on Chinese Mixtral
Yiming Cui, Xin Yao
Mixtral, a representative sparse mixture of experts (SMoE) language model, has received significant attention due to its unique model design and superior performance. Based on Mixt…
MiniRBT: A Two-stage Distilled Small Chinese Pre-trained Model
Xin Yao, Ziqing Yang, Yiming Cui +1
In natural language processing, pre-trained language models have become essential infrastructures. However, these models often suffer from issues such as large size, long inference…
Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca
Yiming Cui, Ziqing Yang, Xin Yao
Large Language Models (LLMs), such as ChatGPT and GPT-4, have dramatically transformed natural language processing research and shown promising strides towards Artificial General I…
Gradient-based Intra-attention Pruning on Pre-trained Language Models
Ziqing Yang, Yiming Cui, Xin Yao +1
Pre-trained language models achieve superior performance but are computationally expensive. Techniques such as pruning and knowledge distillation have been developed to reduce thei…