4 papers · 1 filter
FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy
Qingyao Yang, Runming Yang, He Xiao +7
While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlooks the necessity of specializ…
Simple-OPD: Demystifying Warm-up for On-policy Distillation
Tao Liu, Taiqiang Wu, Mao Zheng +5
On-policy distillation (OPD) trains a student on its own rollouts with token-level supervision from teacher models, but its effectiveness can depend strongly on the warm-up stage b…
Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
Taiqiang Wu, Yuxin Cheng, Chenchen Ding +5
Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy ef…
ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
Tao Liu, Taiqiang Wu, Runming Yang +3
Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many…