8 papers
NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Jiangyong Yu, Xiaomeng Han, Xing Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and c…
FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection
Jiangyong Yu, Changyong Shu, Sifan Zhou +4
Camera-based multi-view 3D detection is crucial for autonomous driving. PETR and its variants (PETRs) excel in benchmarks but face deployment challenges due to high computational c…
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
Xing Hu, Zhixuan Chen, Dawei Yang +5
Mixture-of-Experts (MoE) large language models (LLMs), which leverage dynamic routing and sparse activation to enhance efficiency and scalability, have achieved higher performance…
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization
Chen Xu, Yuxuan Yue, Zukang Xu +6
RWKV is a modern RNN architecture with comparable performance to Transformer, but still faces challenges when deployed to resource-constrained devices. Post Training Quantization (…
FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection
Jiangyong Yu, Changyong Shu, Sifan Zhou +4
Camera-based multi-view 3D detection is crucial for autonomous driving. PETR and its variants (PETRs) excel in benchmarks but face deployment challenges due to high computational c…
MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization
JiangYong Yu, Sifan Zhou, Dawei Yang +7
Multimodal large language models (MLLMs) have garnered widespread attention due to their ability to understand multimodal input. However, their large parameter sizes and substantia…