1.3k citations · 3.6k across the 28 of their papers we have counts for
24 papers · 1 filter
SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation
Qi Yan, Zhengyang Liang, Yang Song +2
Diffusion models based on permutation-equivariant networks can learn permutation-invariant distributions for graph data. However, in comparison to their non-invariant counterparts,…
Density Ratio Estimation via Infinitesimal Classification
Kristy Choi, Chenlin Meng, Yang Song +1
Density ratio estimation (DRE) is a fundamental machine learning technique for comparing two probability distributions. However, existing methods struggle in high-dimensional setti…
Improved Autoregressive Modeling with Distribution Smoothing
Chenlin Meng, Jiaming Song, Yang Song +2
While autoregressive models excel at image compression, their sample quality is often lacking. Although not realistic, generated images often have high likelihood according to the…
How to Train Your Energy-Based Models
Yang Song, Diederik P. Kingma
Energy-Based Models (EBMs), also known as non-normalized probabilistic models, specify probability density or mass functions up to an unknown normalizing constant. Unlike most othe…
Learning Energy-Based Models by Diffusion Recovery Likelihood
Ruiqi Gao, Yang Song, Ben Poole +2
While energy-based models (EBMs) exhibit a number of desirable properties, training and sampling on high-dimensional datasets remains challenging. Inspired by recent progress on di…
Score-Based Generative Modeling through Stochastic Differential Equations
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma +3
Creating noise from data is easy; creating data from noise is generative modeling. We present a stochastic differential equation (SDE) that smoothly transforms a complex data distr…
Autoregressive Score Matching
Chenlin Meng, Lantao Yu, Yang Song +2
Autoregressive models use chain rule to define a joint probability distribution as a product of conditionals. These conditionals need to be normalized, imposing constraints on the…
Imitation with Neural Density Models
Kuno Kim, Akshat Jindal, Yang Song +3
We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) us…
Understanding Classifier Mistakes with Generative Models
Laëtitia Shao, Yang Song, Stefano Ermon
Although deep neural networks are effective on supervised learning tasks, they have been shown to be brittle. They are prone to overfitting on their training distribution and are e…
Efficient Learning of Generative Models via Finite-Difference Score Matching
Tianyu Pang, Kun Xu, Chongxuan Li +3
Several machine learning applications involve the optimization of higher-order derivatives (e.g., gradients of gradients) during training, which can be expensive in respect to memo…
Improved Techniques for Training Score-Based Generative Models
Yang Song, Stefano Ermon
Score-based generative models can produce high quality image samples comparable to GANs, without requiring adversarial optimization. However, existing training procedures are limit…
Gaussianization Flows
Chenlin Meng, Yang Song, Jiaming Song +1
Iterative Gaussianization is a fixed-point iteration procedure that can transform any continuous random vector into a Gaussian one. Based on iterative Gaussianization, we propose a…
Permutation Invariant Graph Generation via Score-Based Generative Modeling
Chenhao Niu, Yang Song, Jiaming Song +3
Learning generative models for graph-structured data is challenging because graphs are discrete, combinatorial, and the underlying data distribution is invariant to the ordering of…
Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
Yusuke Tashiro, Yang Song, Stefano Ermon
Adversarial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e.g., to initialize optimization-based white-box attacks or gener…
Training Deep Energy-Based Models with f-Divergence Minimization
Lantao Yu, Yang Song, Jiaming Song +1
Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging because of the intractable partition function. They are typically…
Unsupervised Out-of-Distribution Detection with Batch Normalization
Jiaming Song, Yang Song, Stefano Ermon
Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to…
Efficient Graph Generation with Graph Recurrent Attention Networks
Renjie Liao, Yujia Li, Yang Song +6
We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nod…
MintNet: Building Invertible Neural Networks with Masked Convolutions
Yang Song, Chenlin Meng, Stefano Ermon
We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible ar…
Generative Modeling by Estimating Gradients of the Data Distribution
Yang Song, Stefano Ermon
We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. Because gradients can b…
Sliced Score Matching: A Scalable Approach to Density and Score Estimation
Yang Song, Sahaj Garg, Jiaxin Shi +1
Score matching is a popular method for estimating unnormalized statistical models. However, it has been so far limited to simple, shallow models or low-dimensional data, due to the…
Constructing Unrestricted Adversarial Examples with Generative Models
Yang Song, Rui Shu, Nate Kushman +1
Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this typ…
Accelerating Natural Gradient with Higher-Order Invariance
Yang Song, Jiaming Song, Stefano Ermon
An appealing property of the natural gradient is that it is invariant to arbitrary differentiable reparameterizations of the model. However, this invariance property requires infin…
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
Yang Song, Taesup Kim, Sebastian Nowozin +2
Adversarial perturbations of normal images are usually imperceptible to humans, but they can seriously confuse state-of-the-art machine learning models. What makes them so special…
Training Deep Neural Networks via Direct Loss Minimization
Yang Song, Alexander G. Schwing, Richard S. Zemel +1
Supervised training of deep neural nets typically relies on minimizing cross-entropy. However, in many domains, we are interested in performing well on metrics specific to the appl…