3 citations · 34 across the 47 of their papers we have counts for
51 papers
Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation
Rui Wang, Ruijie Wang, Bo Chen +2
Self-distillation turns knowledge distillation into a closed learning loop and offers a path toward recursive self-improvement. Without an external teacher, however, the model must…
Can Language Models Compose Skills In-Context?
Zidong Liu, Zhuoyan Xu, Zhenmei Shi +1
Composing basic skills from simple tasks to accomplish composite tasks is crucial for modern intelligent systems. We investigate the in-context composition ability of language mode…
Kernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning
Dechen Zhang, Zhenmei Shi, Yi Zhang +2
Kernel ridge regression (KRR) is a foundational tool in machine learning, with recent work emphasizing its connections to neural networks. However, existing theory primarily addres…
Can You Count to Nine? A Human Evaluation Benchmark for Counting Limits in Modern Text-to-Video Models
Xuyang Guo, Zekai Huang, Jiayan Huo +4
Generative models have driven significant progress in a variety of AI tasks, including text-to-video generation, where models like Video LDM and Stable Video Diffusion can produce…
Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
Yifang Chen, Xiaoyu Li, Yingyu Liang +3
The key-value (KV) cache in the tensor version of transformers presents a significant bottleneck during inference. While previous work analyzes the fundamental space complexity bar…
Theoretical Guarantees for High Order Trajectory Refinement in Generative Flows
Chengyue Gong, Xiaoyu Li, Yingyu Liang +4
Flow matching has emerged as a powerful framework for generative modeling, offering computational advantages over diffusion models by leveraging deterministic Ordinary Differential…
HOFAR: High-Order Augmentation of Flow Autoregressive Transformers
Yingyu Liang, Zhizhou Sha, Zhenmei Shi +2
Flow Matching and Transformer architectures have demonstrated remarkable performance in image generation tasks, with recent work FlowAR [Ren et al., 2024] synergistically integrati…
Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches
Yifang Chen, Xuyang Guo, Xiaoyu Li +3
Recently, Large Language Models (LLMs) have achieved remarkable success. A key factor behind this success is the scaling law observed by OpenAI. Specifically, for models with Trans…
Visual Autoregressive Transformers Must Use Memory
Yang Cao, Xiaoyu Li, Yekun Ke +3
A fundamental challenge in Visual Autoregressive models is the substantial memory overhead required during inference to store previously generated representations. Despite various…
Learning to Inference Adaptively for Multimodal Large Language Models
Zhuoyan Xu, Khoi Duc Nguyen, Preeti Mukherjee +4
Multimodal Large Language Models (MLLMs) have shown impressive capabilities in visual reasoning, yet come with substantial computational cost, limiting their deployment in resource…
Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help
Xuyang Guo, Jiayan Huo, Yingyu Liang +4
Generative modeling is widely regarded as one of the most essential problems in today's AI community, with text-to-image generation having gained unprecedented real-world impacts.…
When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?
Chenyang Li, Yingyu Liang, Zhenmei Shi +1
The weighted low-rank approximation problem is a fundamental numerical linear algebra problem and has many applications in machine learning. Given a weight matrix …
Force Matching with Relativistic Constraints: A Physics-Inspired Approach to Stable and Efficient Generative Modeling
Yang Cao, Bo Chen, Xiaoyu Li +5
This paper introduces Force Matching (ForM), a novel framework for generative modeling that represents an initial exploration into leveraging special relativistic mechanics to enha…
Universal Approximation of Visual Autoregressive Transformers
Yifang Chen, Xiaoyu Li, Yingyu Liang +2
We investigate the fundamental limits of transformer-based foundation models, extending our analysis to include Visual Autoregressive (VAR) transformers. VAR represents a big step…
Dissecting Submission Limit in Desk-Rejections: A Mathematical Analysis of Fairness in AI Conference Policies
Yuefan Cao, Xiaoyu Li, Yingyu Liang +4
As AI research surges in both impact and volume, conferences have imposed submission limits to maintain paper quality and alleviate organizational pressure. In this work, we examin…
High-Order Matching for One-Step Shortcut Diffusion Models
Bo Chen, Chengyue Gong, Xiaoyu Li +5
One-step shortcut diffusion models [Frans, Hafner, Levine and Abbeel, ICLR 2025] have shown potential in vision generation, but their reliance on first-order trajectory supervision…
RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
Yuefan Cao, Chengyue Gong, Xiaoyu Li +4
Text-to-video generation models have made impressive progress, but they still struggle with generating videos with complex features. This limitation often arises from the inability…
On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
Yekun Ke, Xiaoyu Li, Yingyu Liang +3
Recently, Visual Autoregressive () Models introduced a groundbreaking advancement in the field of image generation, offering a scalable approach through a coarse-to-f…
On Computational Limits of FlowAR Models: Expressivity and Efficiency
Yang Cao, Chengyue Gong, Yekun Ke +5
The expressive power and computational complexity of deep visual generative models, such as flow-based and autoregressive (AR) models, have gained considerable interest for their w…
DPBloomfilter: Securing Bloom Filters with Differential Privacy
Yekun Ke, Yingyu Liang, Zhizhou Sha +3
The Bloom filter is a simple yet space-efficient probabilistic data structure that supports membership queries for dramatically large datasets. It is widely utilized and implemente…
On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective
Xiaoyu Li, Yingyu Liang, Zhenmei Shi +3
Graph Neural Networks (GNNs) have become the standard approach for learning and reasoning over relational data, leveraging the message-passing mechanism that iteratively propagates…
Circuit Complexity Bounds for Visual Autoregressive Model
Yekun Ke, Xiaoyu Li, Yingyu Liang +2
Understanding the expressive ability of a specific model is essential for grasping its capacity limitations. Recently, several studies have established circuit complexity bounds fo…
Neural Algorithmic Reasoning for Hypergraphs with Looped Transformers
Zekai Huang, Yingyu Liang, Zhenmei Shi +2
Looped Transformers have shown exceptional neural algorithmic reasoning capability in simulating traditional graph algorithms, but their application to more complex structures like…
Theoretical Constraints on the Expressive Power of -based Tensor Attention Transformers
Xiaoyu Li, Yingyu Liang, Zhenmei Shi +2
Tensor Attention extends traditional attention mechanisms by capturing high-order correlations across multiple modalities, addressing the limitations of classical matrix-based atte…