5 papers
A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
Duo Li, Zuhao Yang, Xiaoqin Zhang +2
Discrete diffusion-based multimodal large language models (dMLLMs) have emerged as a promising alternative to autoregressive MLLMs thanks to their advantages in parallel decoding a…
ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance
Duo Li, Zuhao Yang, Xiaoqin Zhang +2
Visual token pruning aims to compress and prune redundant visual tokens which play a critical role in efficient inference with large vision-language models (LVLMs). However, most e…
DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
Masahiro Tanaka, Du Li, Umesh Chand +3
The rapid growth of deep learning models has increased the demand for efficient distributed training strategies. Fully sharded approaches like ZeRO-3 and FSDP partition model param…
Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
Yifan Pu, Zhuofan Xia, Jiayi Guo +9
This paper identifies significant redundancy in the query-key interactions within self-attention mechanisms of diffusion transformer models, particularly during the early stages of…
MCSD: An Efficient Language Model with Diverse Fusion
Hua Yang, Duohai Li, Shiman Li
Transformers excel in Natural Language Processing (NLP) due to their prowess in capturing long-term dependencies but suffer from exponential resource consumption with increasing se…