2 citations · 2 across the 5 of their papers we have counts for
6 papers
CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
Yixuan Wang, Haoyu Qiao, Lujun Li +2
Large Language Models (LLMs) confront significant memory challenges due to the escalating KV cache with increasing sequence length. As a crucial technique, existing cross-layer KV…
Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
Te Zhang, Yuheng Li, Junxiang Wang +1
Large multimodal models (LMMs) have advanced significantly by integrating visual encoders with extensive language models, enabling robust reasoning capabilities. However, compressi…
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
Lujun Li, Zhu Qiyuan, Jiacheng Wang +4
Mixture of Experts (MoE) LLMs face significant obstacles due to their massive parameter scale, which imposes memory, storage, and deployment challenges. Although recent expert merg…
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
Hao Gu, Lujun Li, Hao Wang +7
Binary quantization represents the most extreme form of compression, reducing weights to +/-1 for maximal memory and computational efficiency. While recent sparsity-aware binarizat…
You Know What I'm Saying: Jailbreak Attack via Implicit Reference
Tianyu Wu, Lingrui Mei, Ruibin Yuan +3
While recent advancements in large language model (LLM) alignment have enabled the effective identification of malicious objectives involving scene nesting and keyword rewriting, o…
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models
Cheng Lin, Lujun Li, Dezhi Li +3
In this paper, we introduce Nested Low-Rank Adaptation (NoRA), a novel approach to parameter-efficient fine-tuning that extends the capabilities of Low-Rank Adaptation (LoRA) techn…