8 papers
SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
Boyan Shi, Wei Chen, Shuyuan Zhao +4
The combination of Mixture-of-Experts (MoE) and Low-Rank Adaptation (LoRA) has shown significant potential for enhancing the multi-task learning capabilities of Large Language Mode…
STK-Adapter: Incorporating Evolving Graph and Event Chain for Temporal Knowledge Graph Extrapolation
Shuyuan Zhao, Wei Chen, Weijie Zhang +6
Temporal Knowledge Graph (TKG) extrapolation aims to predict future events based on historical facts. Recent studies have attempted to enhance TKG extrapolation by integrating TKG'…
Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttention
Mengqi Liao, Lu Wang, Chaoyun Zhang +6
With reasoning becoming the generative paradigm for large language models (LLMs), the memory bottleneck caused by KV cache during the decoding phase has become a critical factor li…
G-KV: Decoding-Time KV Cache Eviction with Global Attention
Mengqi Liao, Lu Wang, Chaoyun Zhang +7
Recent reasoning large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths. KV cache compressio…
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs
Mengqi Liao, Xiangyu Xi, Ruinian Chen +5
Reasoning large language models (LLMs) excel in complex tasks, which has drawn significant attention to reinforcement learning (RL) for LLMs. However, existing approaches allocate…
A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning
Zhiyu Zhang, Wei Chen, Youfang Lin +1
Recent Continual Learning (CL)-based Temporal Knowledge Graph Reasoning (TKGR) methods focus on significantly reducing computational cost and mitigating catastrophic forgetting cau…