5 papers
Smoothing Dark Areas in Molecular Latent Diffusion
Xi Wang, Jiahan Li, Yuxuan Xia +3
Latent diffusion is a promising framework for scalable 3D molecular generation, but it requires a latent space that remains smooth, valid, and navigable beyond posterior samples. E…
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
Xin Teng, Canyu Zhang, Shaoyi Zheng +3
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precomp…
Submodular Context Partitioning and Compression for In-Context Learning
Shaoyi Zheng, Canyu Zhang, Tianyi Zhou +1
In-context learning (ICL) enables efficient few-shot learning in large language models (LLMs) without training, but suffers from the quadratic input complexity of transformers, lim…
HilbertA: Hilbert Attention for Image Generation with Diffusion Models
Shaoyi Zheng, Wenbo Lu, Yuxuan Xia +2
Designing sparse attention for diffusion transformers requires reconciling two-dimensional spatial locality with GPU efficiency, a trade-off that current methods struggle to achiev…
ToMA: Token Merge with Attention for Diffusion Models
Wenbo Lu, Shaoyi Zheng, Yuxuan Xia +1
Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers' quadratic attention complexity. Plug-and-play token reduction methods like…