6 papers
AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse
Jie Ou, Jinyu Guo, Shiyao Guo +5
Many-Shot In-Context Learning (ICL) has emerged as a promising paradigm, leveraging extensive examples to unlock the reasoning potential of Large Language Models (LLMs). However, e…
CAP: Controllable Alignment Prompting for Unlearning in LLMs
Zhaokun Wang, Jinyu Guo, Jingwen Pu +7
Large language models (LLMs) trained on unfiltered corpora inherently risk retaining sensitive information, necessitating selective knowledge unlearning for regulatory compliance a…
GateRA: Token-Aware Modulation for Parameter-Efficient Fine-Tuning
Jie Ou, Shuaihong Jiang, Yingjun Du +1
Parameter-efficient fine-tuning (PEFT) methods, such as LoRA, DoRA, and HiRA, enable lightweight adaptation of large pre-trained models via low-rank updates. However, existing PEFT…
Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
Zhaokun Wang, Jinyu Guo, Jingwen Pu +5
Current parameter-efficient fine-tuning methods for adapting pre-trained language models to downstream tasks are susceptible to interference from noisy data. Conventional noise-han…
HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation
Jinyu Guo, Xunlei Chen, Qiyang Xia +5
Retrieval-Augmented Generation (RAG) encounters efficiency challenges when scaling to massive knowledge bases while preserving contextual relevance. We propose Hash-RAG, a framewor…
Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps
Jie Ou, Jinyu Guo, Shuaihong Jiang +4
Retrieval-augmented generation (RAG) has emerged as a pivotal method for expanding the knowledge of large language models. To handle complex queries more effectively, researchers d…