1 citations · 1 across the 12 of their papers we have counts for
6 papers · 1 filter
Acceptance-Aware Draft Model Training for Speculative Decoding
Tianhua Xia, Mugilan Ganesan, Yifei Feng +3
Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are verified by the target model…
LSVD: Loss-Aware Low-Rank Approximation for Efficient Low-Precision Vision-Language Models
Haiyu Wang, Yutong Wang, Leshu Li +2
Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deployment challenging on resource…
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
Zining Liu, Yunhai Hu, Tianhua Xia +4
Speculative decoding (SD) has proven to be an effective technique for accelerating autoregressive generation in large language models (LLMs) however, its application to vision-lang…
CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
Xiangyang Yin, Xingyu Liu, Tianhua Xia +5
Outliers have emerged as a fundamental bottleneck in preserving accuracy for low-precision large models, particularly within Mixture-of-Experts (MoE) architectures that are increas…
DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
Fengze Yu, Leshu Li, Brad McDanel +1
Large language model (LLM) inference often suffers from high decoding latency and limited scalability across heterogeneous edge-cloud environments. Existing speculative decoding (S…
DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
Jingyang Xiang, Sai Qian Zhang
Rotating the activation and weight matrices to reduce the influence of outliers in large language models (LLMs) has recently attracted significant attention, particularly in the co…