12 citations · 32 across the 14 of their papers we have counts for
11 papers · 1 filter
Group Representational Position Encoding
Yifan Zhang, Zixiang Chen, Yifeng Liu +6
We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multi…
Global Convergence and Rich Feature Learning in -Layer Infinite-Width Neural Networks under P Parametrization
Zixiang Chen, Greg Yang, Qingyue Zhao +1
Despite deep neural networks' powerful representation learning capabilities, theoretical understanding of how networks can simultaneously achieve meaningful feature learning and gl…
Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis
Zikun Zhang, Zixiang Chen, Quanquan Gu
Diffusion models have achieved great success in generating high-dimensional samples across various applications. While the theoretical guarantees for continuous-state diffusion mod…
Matching the Statistical Query Lower Bound for -Sparse Parity Problems with Sign Stochastic Gradient Descent
Yiwen Kou, Zixiang Chen, Quanquan Gu +1
The -sparse parity problem is a classical problem in computational complexity and algorithmic theory, serving as a key benchmark for understanding computational classes. In this…
Guided Discrete Diffusion for Electronic Health Record Generation
Jun Han, Zixiang Chen, Yongqian Li +4
Electronic health records (EHRs) are a pivotal data source that enables numerous applications in computational medicine, e.g., disease progression prediction, clinical trial design…
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation
Huizhuo Yuan, Zixiang Chen, Kaixuan Ji +1
Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI), especially when compared with the remarkable progress made in fine-tun…
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Zixiang Chen, Yihe Deng, Huizhuo Yuan +2
Harnessing the power of human-annotated data through Supervised Fine-Tuning (SFT) is pivotal for advancing Large Language Models (LLMs). In this paper, we delve into the prospect o…
Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time
Zixiang Chen, Huizhuo Yuan, Yongqian Li +3
Discrete diffusion models have emerged as powerful tools for high-quality data generation. Despite their success in discrete spaces, such as text generation tasks, the acceleration…
Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data
Yiwen Kou, Zixiang Chen, Quanquan Gu
The implicit bias towards solutions with favorable properties is believed to be a key reason why neural networks trained by gradient-based optimization can generalize well. While t…
Why Does Sharpness-Aware Minimization Generalize Better Than SGD?
Zixiang Chen, Junkai Zhang, Yiwen Kou +3
The challenge of overfitting, in which the model memorizes the training data and fails to generalize to test data, has become increasingly significant in the training of large neur…
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP
Zixiang Chen, Yihe Deng, Yuanzhi Li +1
Multi-modal learning has become increasingly popular due to its ability to leverage information from different data sources (e.g., text and images) to improve the model performance…