Publications (30)
OneRec Technical Report
Guorui Zhou, Jiaxin Deng, Jinghao Zhang +62
Recommender systems have been widely used in various large-scale user-oriented platforms for many years. However, compared to the rapid developments in the AI community, recommenda…
Kwai Keye-VL-2.0 Technical Report
Kwai Keye Team, Bin Wen, Changyi Liu +50
We introduce Kwai Keye-VL-2.0-30B-A3B, an open-source Mixture-of-Experts (MoE) multimodal foundation model designed to advance long-video understanding and agentic intelligence. To…
Kwai Summary Attention Technical Report
Chenglong Chu, Guorui Zhou, Guowang Zhang +35
Long-context ability, has become one of the most important iteration direction of next-generation Large Language Models, particularly in semantic understanding/reasoning, code agen…
PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching
Qianchao Zhu, Xucheng Ye, Yuliang Liu +2
Mixture-of-Experts models have become a dominant architecture for scaling Large Language Models by activating only a sparse subset of experts per token. However, latency-critical M…
KwaiYiiMath: Technical Report
Jiayi Fu, Lei Lin, Xiaoyang Gao +18
Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on math…
Kwai Keye-VL 1.5 Technical Report
Biao Yang, Bin Wen, Boyang Ding +58
In recent years, the development of Large Language Models (LLMs) has significantly advanced, extending their capabilities to multimodal tasks through Multimodal Large Language Mode…
Kwai Keye-VL Technical Report
Kwai Keye Team, Biao Yang, Bin Wen +57
While Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities on static images, they often fall short in comprehending dynamic, information-dense short-form vi…
Mosaic: Cross-Modal Clustering for Efficient Video Understanding
Tuowei Wang, He Zhou, Chengru Song +2
Large vision-language models (VLMs) are enabling interactive video reasoning, giving rise to streaming long-video understanding. In this setting, frames arrive continuously, while…
Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
Chenxi Sun, Hongzhi Zhang, Zijia Lin +8
Large language models have demonstrated exceptional capability in natural language understanding and generation. However, their generation speed is limited by the inherently sequen…
ScalingAttention: Discovering Intrinsic Sparse Attention Topology for Video Diffusion Transformers
Ruiliang Zhou, Xuecheng Wu, Kang He +6
While Diffusion Transformers (DiTs) have revolutionized high-fidelity video generation, their reliance on 3D full attention creates a quadratic computational bottleneck. Existing s…
X-Stage: An Overlooked Pipeline Stage for Communication-Computation Overlap in DiT Inference
Jianwen Xian, Zhiyuan Xu, Yuchen Li +8
Fine-grained, device-initiated communication lets persistent GPU kernels in distributed diffusion transformer (DiT) inference issue remote stores and overlap data movement with Ten…
OpenOneRec Technical Report
Guorui Zhou, Honghui Bao, Jiaming Huang +44
While the OneRec series has successfully unified the fragmented recommendation pipeline into an end-to-end generative framework, a significant gap remains between recommendation sy…
Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
Junqi Jin, Chengru Song, Han Li +3
Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by…
PA&DA: Jointly Sampling PAth and DAta for Consistent NAS
Shun Lu, Yu Hu, Longxing Yang +4
Based on the weight-sharing mechanism, one-shot NAS methods train a supernet and then inherit the pre-trained weights to evaluate sub-models, largely reducing the search cost. Howe…
SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
Zhouyang Li, Yuliang Liu, Wei Zhang +4
Pipeline Parallelism (PP) serves as a crucial technique for training Large Language Models (LLMs), owing to its capability to alleviate memory pressure from model states with relat…
Deep Interest Network for Click-Through Rate Prediction
Guorui Zhou, Chengru Song, Xiaoqiang Zhu +7
Click-through rate prediction is an essential task in industrial applications, such as online advertising. Recently deep learning based models have been proposed, which follow a si…
HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation
Jinliang Shen, Lianghao Su, Zheming Li +4
Autoregressive (AR) video diffusion models have become a promising paradigm for long and streaming video synthesis, but the continuously growing Key-Value (KV) cache makes attentio…
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Yang Jin, Kun Xu, Liwei Chen +12
Recently, the remarkable advance of the Large Language Model (LLM) has inspired researchers to transfer its extraordinary reasoning capability to both vision and language data. How…
SHARK: A Lightweight Model Compression Approach for Large-scale Recommender Systems
Beichuan Zhang, Chenggen Sun, Jianchao Tan +7
Increasing the size of embedding layers has shown to be effective in improving the performance of recommendation models, yet gradually causing their sizes to exceed terabytes in in…
Kelix Technical Report
Boyang Ding, Chenglong Chu, Dunju Zang +28
Autoregressive large language models (LLMs) scale well by expressing diverse tasks as sequences of discrete natural-language tokens and training with next-token prediction, which u…
USDC: Unified Static and Dynamic Compression for Visual Transformer
Huan Yuan, Chao Liao, Jianchao Tan +5
Visual Transformers have achieved great success in almost all vision tasks, such as classification, detection, and so on. However, the model complexity and the inference speed of t…
: Re-conceptualizing Distribution Matching as a Reward for Diffusion Distillation
Linqian Fan, Peiqin Sun, Tiancheng Wen +2
Diffusion models achieve state-of-the-art generative performance but are fundamentally bottlenecked by their slow, iterative sampling process. While diffusion distillation techniqu…
Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks
Jue Chen, Huan Yuan, Jianchao Tan +3
Brain-inspired Spiking Neural Networks (SNNs) have the characteristics of event-driven and high energy-efficient, which are different from traditional Artificial Neural Networks (A…
OneRec-V2 Technical Report
Guorui Zhou, Hengrui Hu, Hongtao Cheng +72
Recent breakthroughs in generative AI have transformed recommender systems through end-to-end generation. OneRec reformulates recommendation as an autoregressive generation task, a…
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Peng Yao, Chao Liao, Jiyuan Jia +4
Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden…
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
Yang Jin, Zhicheng Sun, Kun Xu +9
In light of recent advances in multimodal Large Language Models (LLMs), there is increasing attention to scaling them from image-text data to more informative real-world videos. Co…
JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models
Ruiyi Ding, Jie Li, He Kang +4
Group Relative Policy Optimization (GRPO) is a powerful reinforcement learning algorithm for aligning generative models with human preferences. While successful in large language m…
VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation
Xinlong Chen, Yuanxing Zhang, Chongling Rao +7
The training of controllable text-to-video (T2V) models relies heavily on the alignment between videos and captions, yet little existing research connects video caption evaluation…
OneReason Technical Report
OneRec Team, Biao Yang, Boyang Ding +81
Generative recommendation models in the OneRec family have been widely deployed in many real-world services, such as short-video, live-streaming, advertising, and e-commerce. Howev…
Latent Reward Registers for Diffusion Preference Alignment
Yuanshen Guan, Zipeng Feng, Zhiwei Xiong +2
Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment…