3 citations · 7 across the 10 of their papers we have counts for
13 papers
Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation
Xiaoyu Wu, Weihang Guo, Yifei Wang +3
Standard video generators do not natively compact historical context into reusable memory tokens. As generation continues, the growing history makes it increasingly difficult to re…
Representation by Design in Generation: Cross-View Class-Token Alignment in Diffusion Transformers
Xiaoyu Wu, Yifei Wang, Chen Wei
Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas the models' own representations a…
Taming Outlier Tokens in Diffusion Transformers
Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu +3
We study outlier tokens in Diffusion Transformers (DiTs) for image generation. Prior work has shown that Vision Transformers (ViTs) can produce a small number of high-norm tokens t…
Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning
Yuxuan Gu, Weimin Bai, Yifei Wang +2
Masked auto-regressive diffusion models (MAR) benefit from the expressive modeling ability of diffusion models and the flexibility of masked auto-regressive ordering. However, vani…
Language Ranker: A Lightweight Ranking framework for LLM Decoding
Chenheng Zhang, Tianqi Du, Jizhe Zhang +4
Conventional research on large language models (LLMs) has primarily focused on refining output distributions, while paying less attention to the decoding process that transforms th…
Biologically Plausible Learning via Bidirectional Spike-Based Distillation
Changze Lv, Yifei Wang, Yanxun Zhang +7
Developing biologically plausible learning algorithms that can achieve performance comparable to error backpropagation remains a longstanding challenge. Existing approaches often c…
Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction
Yifei Wang, Weimin Bai, Colin Zhang +3
In this paper, we unify more than 10 existing one-step diffusion distillation approaches, such as Diff-Instruct, DMD, SIM, SiD, -distill, etc, inside a theory-driven framework w…
Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing
Nanyang Ye, Qiao Sun, Yifei Wang +10
Analog computing using non-volatile memristors has emerged as a promising solution for energy-efficient deep learning. New materials, like perovskites-based memristors are recently…
Integrating Amortized Inference with Diffusion Models for Learning Clean Distribution from Corrupted Images
Yifei Wang, Weimin Bai, Weijian Luo +2
Diffusion models (DMs) have emerged as powerful generative models for solving inverse problems, offering a good approximation of prior distributions of real-world image data. Typic…
On the Role of Discrete Tokenization in Visual Representation Learning
Tianqi Du, Yifei Wang, Yisen Wang
In the realm of self-supervised learning (SSL), masked image modeling (MIM) has gained popularity alongside contrastive learning methods. MIM involves reconstructing masked regions…
An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations
Weimin Bai, Yifei Wang, Wenzheng Chen +1
Diffusion models excel in solving imaging inverse problems due to their ability to model complex image priors. However, their reliance on large, clean datasets for training limits…
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
Qi Zhang, Tianqi Du, Haotian Huang +2
In recent years, the rise of generative self-supervised learning (SSL) paradigms has exhibited impressive performance across visual, language, and multi-modal domains. While the va…
Do Generated Data Always Help Contrastive Learning?
Yifei Wang, Jizhe Zhang, Yisen Wang
Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augment…