17 citations · 42 across the 27 of their papers we have counts for
29 papers
NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks
Zhangming Li, Qinghao Hu, Weifan Guan +3
Full fine-tuning has achieved remarkable success on reasoning-intensive tasks, but it often requires substantial GPU resource consumption. Parameter-efficient fine-tuning (PEFT) pr…
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Yuanteng Chen, Zhilei Liu, Peisong Wang +7
Quantization-aware distillation (QAD) restores much of the short-form question-answering performance lost to sub-3-bit quantization, yet leaves mathematical and code reasoning subs…
You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs
Yuanteng Chen, Qiwei Lai, Chen Tianqi +7
Fine-grained mixture-of-experts (MoE) architectures have become a mainstream design for open-weight LLMs, with hundreds of experts and increasingly many selected per token. This sh…
From Splats to Silicon: Rethinking Computational Efficiency of 3DGS
Minnan Pei, Qiwei Dong, Yihan Zhou +7
3D Gaussian splatting (3DGS) represents scenes with explicit primitives and supports real-time novel-view synthesis, yet its system efficiency varies substantially across scenes, v…
BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression
Yuantian Shao, Peisong Wang, Zhilei Liu +6
Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainl…
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Yuanteng Chen, Nanxin Zeng, Peisong Wang +9
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods…
Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation
Changwang Mei, Peisong Wang, Zekun Li +7
Visual Autoregressive (VAR) models deliver high-quality image generation but suffer from significant inference latency at high resolutions. Recent acceleration approaches most rely…
SparVAR: Exploring Sparsity in Visual AutoRegressive Modeling for Training-Free Acceleration
Zekun Li, Ning Wang, Tongxin Bai +4
Visual AutoRegressive (VAR) modeling has garnered significant attention for its innovative next-scale prediction paradigm. However, mainstream VAR paradigms attend to all tokens ac…
DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs
Zeyu Zhu, Gang Li, Peisong Wang +5
Mixture of Experts (MoE) architectures significantly enhance the capacity of LLMs without proportional increases in computation, but at the cost of a vast parameter size. Offloadin…
IntraSlice: Towards High-Performance Structural Pruning with Block-Intra PCA for LLMs
Meng Li, Peisong Wang, Yuantian Shao +5
Large Language Models (LLMs) achieve strong performance across diverse tasks but face deployment challenges due to their massive size. Structured pruning offers acceleration benefi…
Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE
Yuanteng Chen, Peisong Wang, Nanxin Zeng +5
Test-time scaling improves LLM performance by generating multiple candidate solutions, yet token-level sampling requires temperature tuning that trades off diversity against stabil…
Block Rotation is All You Need for MXFP4 Quantization
Yuantian Shao, Peisong Wang, Yuanteng Chen +3
Large language models (LLMs) have achieved remarkable success, but their rapidly growing scale imposes prohibitive costs in memory, computation, and energy. Post-training quantizat…
DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization
Yuantian Shao, Yuanteng Chen, Peisong Wang +5
Quantization plays a crucial role in accelerating the inference of large-scale models, and rotational matrices have been shown to effectively improve quantization performance by sm…
HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
Zhinan Xie, Peisong Wang, Shuang Qiu +1
Speculative decoding has proven effective for accelerating inference in Large Language Models (LLMs), yet its extension to Vision-Language Models (VLMs) remains limited by the comp…
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
Yuanteng Chen, Peisong Wang, Yuantian Shao +3
Sparse Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs) efficiently. Recent fine-grained MoE designs introduce hundreds of experts pe…
EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
Yuanteng Chen, Yuantian Shao, Peisong Wang +1
Mixture-of-Experts (MoE) has demonstrated promising potential in scaling LLMs. However, it is hindered by two critical challenges: (1) substantial GPU memory consumption to load al…
GCC: A 3DGS Inference Architecture with Gaussian-Wise and Cross-Stage Conditional Processing
Minnan Pei, Gang Li, Junwen Si +6
3D Gaussian Splatting (3DGS) has emerged as a leading neural rendering technique for high-fidelity view synthesis, prompting the development of dedicated 3DGS accelerators for reso…
Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation
Xingting Yao, Qinghao Hu, Fei Zhou +4
Multi-bit spiking neural networks (SNNs) have recently become a heated research spot, pursuing energy-efficient and high-accurate AI. However, with more bits involved, the associat…
FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing
Yingying Deng, Xiangyu He, Changwang Mei +2
Though Rectified Flows (ReFlows) with distillation offers a promising way for fast sampling, its fast inversion transforms images back to structured noise for recovery and followin…
FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
Zeyu Zhu, Peisong Wang, Qinghao Hu +3
Graph Neural Networks (GNNs) have shown great superiority on non-Euclidean graph data, achieving ground-breaking performance on various graph-related tasks. As a practical solution…
TernaryLLM: Ternarized Large Language Model
Tianqi Chen, Zhe Li, Weixiang Xu +6
Large language models (LLMs) have achieved remarkable performance on Natural Language Processing (NLP) tasks, but they are hindered by high computational costs and memory requireme…
Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report
Marcos V. Conde, Radu Timofte, Yibin Huang +40
Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP). Numerous low-level…
Soft Threshold Ternary Networks
Weixiang Xu, Xiangyu He, Tianli Zhao +3
Large neural networks are difficult to deploy on mobile devices because of intensive computation and storage. To alleviate it, we study ternarization, a balance between efficiency…
Differentially Private Federated Learning with Local Regularization and Sparsification
Anda Cheng, Peisong Wang, Xi Sheryl Zhang +1
User-level differential privacy (DP) provides certifiable privacy guarantees to the information that is specific to any user's data in federated learning. Existing methods that ens…