9 papers
HSMLA: Hierarchical Softmax Multi-scale Linear Attention for Efficient Vision Transformers
Dong Liu, Yanxuan Yu, Renata Borovica-Gajic +2
Vision transformers face significant computational overheads in high-resolution dense prediction due to the quadratic complexity of self-attention. Linear attention offers efficien…
RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification
Yanxuan Yu, Dong Liu, Dong liu +11
Class imbalance poses a fundamental challenge in risk-sensitive applications such as fraud detection and medical diagnosis, where minority-class samples are scarce yet critical for…
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Hui Lu, Zheng Chai, Shipeng Bai +15
Driven by scaling laws, recommender systems increasingly rely on larger-scale models to capture complex feature interactions and user behaviors, but this trend also leads to prohib…
PiKV: KV Cache Management System for Mixture of Experts
Dong Liu, Yanxuan Yu, Ben Lengerich +1
As large-scale language models continue to scale up in both size and context length, the memory and communication cost of key-value (KV) cache storage has become a major bottleneck…
AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers
Dong Liu, Yanxuan Yu, Ben Lengerich +1
Diffusion Transformers (DiTs) achieve state-of-the-art performance in high-fidelity image and video generation but suffer from expensive inference due to their iterative denoising…
Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
Dong Liu, Yanxuan Yu, Ying Nian Wu
The success of large language models (LLMs) across diverse NLP tasks has elevated the importance of reasoning chain optimization as a critical step in aligning model behavior with…