activity
20222026
most citedDeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

8 citations · 9 across the 6 of their papers we have counts for

collaborators
Showing 2024Show all

2 papers · 1 filter

cs.CV2024

Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators

Yifan Pu, Zhuofan Xia, Jiayi Guo +9

This paper identifies significant redundancy in the query-key interactions within self-attention mechanisms of diffusion transformer models, particularly during the early stages of…

cs.CL2024

MCSD: An Efficient Language Model with Diverse Fusion

Hua Yang, Duohai Li, Shiman Li

Transformers excel in Natural Language Processing (NLP) due to their prowess in capturing long-term dependencies but suffer from exponential resource consumption with increasing se…