4 citations · 6 across the 5 of their papers we have counts for
7 papers
Preserving Updates in Low-Bit Recurrent State Quantization
Haotian Li, Liran Dong, Kaicheng Xiao +1
Linear attention and recurrent models compress past tokens into a fixed-size state that is updated at every step. However, storing these states in high precision incurs substantial…
Bridging KV-Cache Quantization and Linear Attention: From Theory to Pretrained Weight Migration
Kaicheng Xiao, Liran Dong, Haotian Li +1
KV-cache quantization and linear attention are two representative approaches to tackling the storage and computational costs of Transformers. KV-cache quantization compresses indiv…
SeekRBP: Leveraging Sequence-Structure Integration with Reinforcement Learning for Receptor-Binding Protein Identification
Xiling Luo, Le Ou-Yang, Yang Shen +6
Motivation: Receptor-binding proteins (RBPs) initiate viral infection and determine host specificity, serving as key targets for phage engineering and therapy. However, the identif…
RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State
Kaicheng Xiao, Haotian Li, Liran Dong +1
Linear attention offers an efficient alternative to full attention with a fixed-size recurrent state. However, this state is shared by all tokens, so information from distinct toke…
Improving the Robustness of Large Language Models via Consistency Alignment
Yukun Zhao, Lingyong Yan, Weiwei Sun +6
Large language models (LLMs) have shown tremendous success in following user instructions and generating helpful responses. Nevertheless, their robustness is still far from optimal…
VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization
Dongsheng Zhu, Xunzhu Tang, Weidong Han +5
This paper presents VisLingInstruct, a novel approach to advancing Multi-Modal Language Models (MMLMs) in zero-shot learning. Current MMLMs show impressive zero-shot abilities in m…
Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method
Yukun Zhao, Lingyong Yan, Weiwei Sun +6
Large Language Models (LLMs) have shown great potential in Natural Language Processing (NLP) tasks. However, recent literature reveals that LLMs generate nonfactual responses inter…