7 citations · 11 across the 16 of their papers we have counts for
13 papers · 1 filter
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Yuanteng Chen, Zhilei Liu, Peisong Wang +7
Quantization-aware distillation (QAD) restores much of the short-form question-answering performance lost to sub-3-bit quantization, yet leaves mathematical and code reasoning subs…
You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs
Yuanteng Chen, Qiwei Lai, Chen Tianqi +7
Fine-grained mixture-of-experts (MoE) architectures have become a mainstream design for open-weight LLMs, with hundreds of experts and increasingly many selected per token. This sh…
BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression
Yuantian Shao, Peisong Wang, Zhilei Liu +6
Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainl…
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Yuanteng Chen, Nanxin Zeng, Peisong Wang +9
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods…
IntraSlice: Towards High-Performance Structural Pruning with Block-Intra PCA for LLMs
Meng Li, Peisong Wang, Yuantian Shao +5
Large Language Models (LLMs) achieve strong performance across diverse tasks but face deployment challenges due to their massive size. Structured pruning offers acceleration benefi…
Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE
Yuanteng Chen, Peisong Wang, Nanxin Zeng +5
Test-time scaling improves LLM performance by generating multiple candidate solutions, yet token-level sampling requires temperature tuning that trades off diversity against stabil…
Block Rotation is All You Need for MXFP4 Quantization
Yuantian Shao, Peisong Wang, Yuanteng Chen +3
Large language models (LLMs) have achieved remarkable success, but their rapidly growing scale imposes prohibitive costs in memory, computation, and energy. Post-training quantizat…
DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization
Yuantian Shao, Yuanteng Chen, Peisong Wang +5
Quantization plays a crucial role in accelerating the inference of large-scale models, and rotational matrices have been shown to effectively improve quantization performance by sm…
HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
Zhinan Xie, Peisong Wang, Shuang Qiu +1
Speculative decoding has proven effective for accelerating inference in Large Language Models (LLMs), yet its extension to Vision-Language Models (VLMs) remains limited by the comp…
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
Yuanteng Chen, Peisong Wang, Yuantian Shao +3
Sparse Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs) efficiently. Recent fine-grained MoE designs introduce hundreds of experts pe…
EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
Yuanteng Chen, Yuantian Shao, Peisong Wang +1
Mixture-of-Experts (MoE) has demonstrated promising potential in scaling LLMs. However, it is hindered by two critical challenges: (1) substantial GPU memory consumption to load al…
FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
Zeyu Zhu, Peisong Wang, Qinghao Hu +3
Graph Neural Networks (GNNs) have shown great superiority on non-Euclidean graph data, achieving ground-breaking performance on various graph-related tasks. As a practical solution…
TernaryLLM: Ternarized Large Language Model
Tianqi Chen, Zhe Li, Weixiang Xu +6
Large language models (LLMs) have achieved remarkable performance on Natural Language Processing (NLP) tasks, but they are hindered by high computational costs and memory requireme…