3 citations · 3 across the 10 of their papers we have counts for
4 papers · 1 filter
On the Position Bias of On-Policy Distillation
Yan Xie, Sijie Zhu, Tiansheng Wen +2
On-Policy Distillation (OPD) improves the learning efficiency of standard reinforcement learning through dense, token-level supervision from teachers. In the standard KL objective…
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval
Lixuan Guo, Yifei Wang, Tiansheng Wen +3
Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, thi…
Scaling Attention via Feature Sparsity
Yan Xie, Tiansheng Wen, Tangda Huang +4
Scaling Transformers to ultra-long contexts is bottlenecked by the cost of self-attention. Existing methods reduce this cost along the sequence axis through local window…
CSRv2: Unlocking Ultra-Sparse Embeddings
Lixuan Guo, Yifei Wang, Tiansheng Wen +5
In the era of large foundation models, the quality of embeddings has become a central determinant of downstream task performance and overall system capability. Yet widely used dens…