9 papers
SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging
Shunlong Wu, Hai Lin, Shaoshen Chen +5
Existing KV cache compression methods generally operate on discrete tokens or non-semantic chunks. However, such approaches often lead to semantic fragmentation, where linguistical…
Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
Jiwei Tang, Shilei Liu, Zhicheng Zhang +9
Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computation…
Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
Yangning Li, Tingwei Lu, Yinghui Li +6
Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data…
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment
Jiwei Tang, Zhicheng Zhang, Shunlong Wu +8
Large Language Models (LLMs) have achieved remarkable performance across a wide range of Natural Language Processing (NLP) tasks. However, in long-context scenarios, they face two…
From Token to Line: Enhancing Code Generation with a Long-Term Perspective
Tingwei Lu, Yangning Li, Liyuan Wang +6
The emergence of large language models (LLMs) has significantly promoted the development of code generation task, sparking a surge in pertinent literature. Current research is hind…
RAISE: Reinforced Adaptive Instruction Selection For Large Language Models
Qingsong Lv, Yangning Li, Zihua Lan +8
In the instruction fine-tuning of large language models (LLMs), it is widely recognized that a few high-quality instructions are superior to a large number of low-quality instructi…
Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios
Jiwei Tang, Jin Xu, Tingwei Lu +4
Large language models (LLMs) demonstrate exceptional capabilities in various scenarios. However, they suffer from much redundant information and are sensitive to the position of ke…
UltraWiki: Ultra-fine-grained Entity Set Expansion with Negative Seed Entities
Yangning Li, Qingsong Lv, Tianyu Yu +5
Entity Set Expansion (ESE) aims to identify new entities belonging to the same semantic class as the given set of seed entities. Traditional methods solely relied on positive seed…
MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities
Yangning Li, Tingwei Lu, Yinghui Li +5
The Entity Set Expansion (ESE) task aims to expand a handful of seed entities with new entities belonging to the same semantic class. Conventional ESE methods are based on mono-mod…