6 papers · 1 filter
Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
Taiqiang Wu, Yuxin Cheng, Chenchen Ding +5
Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy ef…
Timber: Training-free Instruct Model Refining with Base via Effective Rank
Taiqiang Wu, Runming Yang, Tao Liu +3
Post-training, which elicits a pretrained Base model into the corresponding Instruct model, is widely considered to be superficial. In this work, we first reinforce this hypothesis…
Shadow-FT: Tuning Instruct Model via Training on Paired Base Model
Taiqiang Wu, Runming Yang, Jiayi Li +4
Large language models (LLMs) consistently benefit from further fine-tuning on various tasks. However, we observe that directly tuning the Instruct (i.e., instruction-tuned) models…
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
Zhen Li, Yupeng Su, Runming Yang +5
Large language models have achieved significant advancements in complex mathematical reasoning benchmarks, such as MATH. However, their substantial computational requirements prese…
LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
Runming Yang, Taiqiang Wu, Jiahao Wang +4
Knowledge distillation (KD) has been a predominant method for compressing Large Language Models (LLMs). In this paper, we first revisit KD and Low-Rank Adaption (LoRA) and demonstr…
LoCa: Logit Calibration for Knowledge Distillation
Runming Yang, Taiqiang Wu, Yujiu Yang
Knowledge Distillation (KD), aiming to train a better student model by mimicking the teacher model, plays an important role in model compression. One typical way is to align the ou…