9 papers
Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models
Yongqin Zeng, Sicheng Pan, Jiale Wang +4
Sparsely activated Mixture-of-Experts (MoE) language models contain substantial structured redundancy among routed experts, but pruning them without downstream calibration data rem…
SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging
Shunlong Wu, Hai Lin, Shaoshen Chen +5
Existing KV cache compression methods generally operate on discrete tokens or non-semantic chunks. However, such approaches often lead to semantic fragmentation, where linguistical…
Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation
Xiaodong Cai, Hai Lin, Shaoxiong Zhan +5
Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring ex…
LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentation
Shaoxiong Zhan, Hai Lin, Hongming Tan +6
As queries in retrieval-augmented generation (RAG) pipelines powered by large language models (LLMs) become increasingly complex and diverse, dense retrieval models have demonstrat…
CLGEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction
Shang Qin, Jingheng Ye, Yinghui Li +5
The growing demand for automated writing assistance in diverse academic domains highlights the need for robust Chinese Grammatical Error Correction (CGEC) systems that can adapt ac…
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment
Jiwei Tang, Zhicheng Zhang, Shunlong Wu +8
Large Language Models (LLMs) have achieved remarkable performance across a wide range of Natural Language Processing (NLP) tasks. However, in long-context scenarios, they face two…
RAISE: Reinforced Adaptive Instruction Selection For Large Language Models
Qingsong Lv, Yangning Li, Zihua Lan +8
In the instruction fine-tuning of large language models (LLMs), it is widely recognized that a few high-quality instructions are superior to a large number of low-quality instructi…
Exploring the Implicit Semantic Ability of Multimodal Large Language Models: A Pilot Study on Entity Set Expansion
Hebin Wang, Yangning Li, Yinghui Li +3
The rapid development of multimodal large language models (MLLMs) has brought significant improvements to a wide range of tasks in real-world applications. However, LLMs still exhi…
CLEME2.0: Towards Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction
Jingheng Ye, Zishan Xu, Yinghui Li +9
The paper focuses on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, which received little attention in previous studies. To bridge the gap, we intro…