3 citations · 3 across the 3 of their papers we have counts for
5 papers
LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation
Zhengqiang Zhang, Lingchen Sun, Rongyuan Wu +4
Latent Diffusion Models (LDMs) typically adopt a two-stage pipeline: an auto-encoder (AE) is first pre-trained to define a latent space, then a diffusion model is trained to perfor…
BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers
Chaodong Xiao, Zhengqiang Zhang, Lei Zhang
Transformers have achieved widespread and remarkable success, while the computational complexity of their attention modules remains a major bottleneck for vision tasks. Existing me…
Pretraining A Large Language Model using Distributed GPUs: A Memory-Efficient Decentralized Paradigm
Jinrui Zhang, Chaodong Xiao, Aoqi Wu +2
Pretraining large language models (LLMs) typically requires centralized clusters with thousands of high-memory GPUs (e.g., H100/A100). Recent decentralized training methods reduce…
Polyline Path Masked Attention for Vision Transformer
Zhongchen Zhao, Chaodong Xiao, Hui Lin +3
Global dependency modeling and spatial position modeling are two core issues of the foundational architecture design in current deep learning frameworks. Recently, Vision Transform…
Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusion
Chaodong Xiao, Minghan Li, Zhengqiang Zhang +2
Selective state space models (SSMs), such as Mamba, highly excel at capturing long-range dependencies in 1D sequential data, while their applications to 2D vision tasks still face…