6 papers
MemoryFormer: Minimize Transformer Computation by Removing Fully-Connected Layers
Ning Ding, Yehui Tang, Haochen Qin +6
In order to reduce the computational complexity of large language models, great efforts have been made to to improve the efficiency of transformer models such as linear attention a…
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
Zhongqian Fu, Tianyi Zhao, Ning Ding +4
Mixture-of-Experts (MoE) models enable scalable computation and performance in large-scale deep learning but face quantization challenges due to sparse expert activation and dynami…
Inference-Time Alignment Control for Diffusion Models with Reinforcement Learning Guidance
Luozhijie Jin, Zijie Qiu, Jie Liu +5
Denoising-based generative models, particularly diffusion and flow matching algorithms, have achieved remarkable success. However, aligning their output distributions with complex…
Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networks
Ni Ding, Lei He, Shengbo Eben Li +1
End-to-end autonomous driving has emerged as a dominant paradigm, yet its highly entangled black-box models pose significant challenges in terms of interpretability and safety assu…
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
Ning Ding, Jing Han, Yuchuan Tian +3
Diffusion Transformer (DiT) has now become the preferred choice for building image generation models due to its great generation capability. Unlike previous convolution-based UNet…
GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task
Ning Ding, Yehui Tang, Zhongqian Fu +3
The upsurge in pre-trained large models started by ChatGPT has swept across the entire deep learning community. Such powerful models demonstrate advanced generative ability and mul…