6 papers · 1 filter
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
Haoyu Wang, Xingyu Yu, Haiyan Zhao +2
Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). Current QAT methods are mainly based on scalar quantization (SQ), which enables e…
UniSVQ: 2-bit Unified Scalar-Vector Quantization
Haoyu Wang, Haiyan Zhao, Xingyu Yu +4
Post-training quantization at the 2-bit level enables low-cost deployment and inference acceleration for large language models (LLMs). Scalar quantization (SQ) and vector quantizat…
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
Jingyi He, Haiyan Zhao, Ruxue Shi +4
Although Sparse Autoencoders (SAEs) have mitigated the opacity of large language models (LLMs) by decomposing dense representations into sparse features, explaining these features…
Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
Haiyan Zhao, Zirui He, Guanchu Wang +3
Activation verbalization explains hidden representations in natural language, but existing methods are mostly limited to self-explanation, where each model explains only its own ac…
DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities
Xiangyu Lu, Wang Xu, Haoyu Wang +5
Real-time speech conversation is essential for natural and efficient human-machine interactions, requiring duplex and streaming capabilities. Traditional Transformer-based conversa…
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
Pengcheng Huang, Zhenghao Liu, Yukun Yan +8
Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence. However, they remain susceptib…