3 papers
cs.CL2025
Submodular Context Partitioning and Compression for In-Context Learning
Shaoyi Zheng, Canyu Zhang, Tianyi Zhou +1
In-context learning (ICL) enables efficient few-shot learning in large language models (LLMs) without training, but suffers from the quadratic input complexity of transformers, lim…
cs.AI2025
HilbertA: Hilbert Attention for Image Generation with Diffusion Models
Shaoyi Zheng, Wenbo Lu, Yuxuan Xia +2
Designing sparse attention for diffusion transformers requires reconciling two-dimensional spatial locality with GPU efficiency, a trade-off that current methods struggle to achiev…
cs.LG2025
ToMA: Token Merge with Attention for Diffusion Models
Wenbo Lu, Shaoyi Zheng, Yuxuan Xia +1
Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers' quadratic attention complexity. Plug-and-play token reduction methods like…