7 papers
PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers
Haopeng Li, Shitong Shao, Wenliang Zhong +4
Diffusion Transformers are fundamental for video and image generation, but their efficiency is bottlenecked by the quadratic complexity of attention. While block sparse attention a…
Blend the Separated: Mixture of Synergistic Experts for Data-Scarcity Drug-Target Interaction Prediction
Xinlong Zhai, Chunchen Wang, Ruijia Wang +7
Drug-target interaction prediction (DTI) is essential in various applications including drug discovery and clinical application. There are two perspectives of input data widely use…
CoRe^2: Collect, Reflect and Refine to Generate Better and Faster
Shitong Shao, Zikai Zhou, Dian Xie +4
Making text-to-image (T2I) generative model sample both fast and well represents a promising research direction. Previous studies have typically focused on either enhancing the vis…
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits
Zikai Zhou, Qizheng Zhang, Hermann Kumbong +1
Fine-tuning large language models (LLMs) is increasingly costly as models scale to hundreds of billions of parameters, and even parameter-efficient fine-tuning (PEFT) methods like…
Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection
Lichen Bai, Shitong Shao, Zikai Zhou +4
Diffusion models, the most popular generative paradigm so far, can inject conditional information into the generation path to guide the latent towards desired directions. However,…
Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer
Shitong Shao, Zikai Zhou, Tian Ye +3
Text-to-image diffusion models (DMs) develop at an unprecedented pace, supported by thorough theoretical exploration and empirical analysis. Unfortunately, the discrepancy between…