3 citations · 7 across the 19 of their papers we have counts for
6 papers · 1 filter
A Theoretical Framework for Masked Pretraining (MPT)
Qi Zhang, Runyu Zhou, Yifei Wang +1
Recently, Masked Pretraining (MPT) based on reconstruction pretraining tasks has risen to a promising self-supervised learning paradigm across various domains and achieves remarkab…
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Tianqi Du, Qi Zhang, Yifei Wang +1
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on obs…
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval
Lixuan Guo, Yifei Wang, Tiansheng Wen +3
Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, thi…
Optimus: Elastic Decoding for Efficient Diffusion LLM Serving
Chiyue Wei, Cong Guo, Bowen Duan +7
Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential exec…
Scaling Attention via Feature Sparsity
Yan Xie, Tiansheng Wen, Tangda Huang +4
Scaling Transformers to ultra-long contexts is bottlenecked by the cost of self-attention. Existing methods reduce this cost along the sequence axis through local window…
StreamingEval: A Unified Evaluation Protocol towards Realistic Streaming Video Understanding
Guowei Tang, Tianwen Qian, Huanran Zheng +2
Real-time, continuous understanding of visual signals is essential for real-world interactive AI applications, and poses a fundamental system-level challenge. Existing research on…