14 citations · 23 across the 14 of their papers we have counts for
5 papers · 1 filter
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
Zhiyuan Shi, Qibo Qiu, Feng Xue +5
The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods often fail to preserve globally…
Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach
Sinan Fan, Liang Xie, Chen Shen +7
Prompt-tuning (PT) for large language models (LLMs) can facilitate the performance on various conventional NLP tasks with significantly fewer trainable parameters. However, our inv…
Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control
Yuxin Xiao, Chaoqun Wan, Yonggang Zhang +5
As the development and application of Large Language Models (LLMs) continue to advance rapidly, enhancing their trustworthiness and aligning them with human preferences has become…
SciPIP: An LLM-based Scientific Paper Idea Proposer
Wenxiao Wang, Lihui Gu, Liye Zhang +7
The rapid advancement of large language models (LLMs) has opened new possibilities for automating the proposal of innovative scientific ideas. This process involves two key phases:…
Model Compression and Efficient Inference for Large Language Models: A Survey
Wenxiao Wang, Wei Chen, Yicong Luo +6
Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it chal…