1 citations · 2 across the 4 of their papers we have counts for
5 papers
Virtual Head Attention
Xibo Ding, Guoxia Wang, Jinle Zeng +4
The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FLOPs, and model quality. GQA ac…
RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
Siran Liu, Guoxia Wang, Sa Wang +7
The quadratic complexity of attention mechanisms poses a critical bottleneck for large language models processing long contexts. While dynamic sparse attention methods offer input-…
Hierarchical LoRA MoE for Efficient CTR Model Scaling
Zhichen Zeng, Mengyue Hang, Xiaolong Liu +11
Deep models have driven significant advances in click-through rate (CTR) prediction. While vertical scaling via layer stacking improves model expressiveness, the layer-by-layer seq…
A Collaborative Ensemble Framework for CTR Prediction
Xiaolong Liu, Zhichen Zeng, Xiaoyi Liu +13
Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into…
InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction
Zhichen Zeng, Xiaolong Liu, Mengyue Hang +25
Click-through rate (CTR) prediction, which predicts the probability of a user clicking an ad, is a fundamental task in recommender systems. The emergence of heterogeneous informati…