288 citations · 2.1k across the 42 of their papers we have counts for
22 papers · 1 filter
Understanding Robustness in Teacher-Student Setting: A New Perspective
Zhuolin Yang, Zhaoxi Chen, Tiffany Cai +3
Adversarial examples have appeared as a ubiquitous property of machine learning models where bounded adversarial perturbation could mislead the models to make arbitrarily incorrect…
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Yuandong Tian, Xinlei Chen, Surya Ganguli
While contrastive approaches of self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs)…
Multi-Agent Collaboration via Reward Attribution Decomposition
Tianjun Zhang, Huazhe Xu, Xiaolong Wang +4
Recent advances in multi-agent reinforcement learning (MARL) have achieved super-human performance in games like Quake 3 and Dota 2. Unfortunately, these techniques require orders-…
Understanding Self-supervised Learning with Dual Deep Networks
Yuandong Tian, Lantao Yu, Xinlei Chen +1
We propose a novel theoretical framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks (e.g., SimCLR). First, we pro…
Joint Policy Search for Multi-agent Collaboration with Imperfect Information
Yuandong Tian, Qucheng Gong, Tina Jiang
To learn good joint policies for multi-agent collaboration with imperfect information remains a fundamental challenge. While for two-player zero-sum games, coordinate-ascent approa…
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
Linnan Wang, Rodrigo Fonseca, Yuandong Tian
High dimensional black-box optimization has broad applications but remains a challenging problem to solve. Given a set of samples $\{\vx_i, y_i\}$, building a global model (like Ba…
Few-shot Neural Architecture Search
Yiyang Zhao, Linnan Wang, Yuandong Tian +2
Efficient evaluation of a network architecture drawn from a large search space remains a key challenge in Neural Architecture Search (NAS). Vanilla NAS evaluates each architecture…
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension
Yuandong Tian
We consider a deep ReLU / Leaky ReLU student network trained from the output of a fixed teacher network of the same depth, with Stochastic Gradient Descent (SGD). The student netwo…
Luck Matters: Understanding Training Dynamics of Deep ReLU Networks
Yuandong Tian, Tina Jiang, Qucheng Gong +1
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship betw…
Sample-Efficient Neural Architecture Search by Learning Action Space
Linnan Wang, Saining Xie, Teng Li +2
Neural Architecture Search (NAS) has emerged as a promising technique for automatic neural network design. However, existing MCTS based NAS approaches often utilize manually design…
Learning to Perform Local Rewriting for Combinatorial Optimization
Xinyun Chen, Yuandong Tian
Search-based methods for hard combinatorial optimization are often guided by heuristics. Tuning heuristics in various conditions and situations is often time-consuming. In this pap…
Learning and Planning with a Semantic Model
Yi Wu, Yuxin Wu, Aviv Tamar +3
Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI. This paper describes progresses on this cha…
A theoretical framework for deep locally connected ReLU network
Yuandong Tian
Understanding theoretical properties of deep and locally connected nonlinear network, such as deep convolutional neural network (DCNN), is still a hard problem despite its empirica…
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
Yuping Luo, Huazhe Xu, Yuanzhi Li +3
Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding…
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
Linnan Wang, Yiyang Zhao, Yuu Jinnai +2
Neural Architecture Search (NAS) has shown great success in automating the design of neural networks, but the prohibitive amount of computations behind current NAS methods requires…
Building Generalizable Agents with a Realistic and Rich 3D Environment
Yi Wu, Yuxin Wu, Georgia Gkioxari +1
Teaching an agent to navigate in an unseen 3D environment is a challenging task, even in the event of simulated environments. To generalize to unseen environments, an agent needs t…
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
Simon S. Du, Jason D. Lee, Yuandong Tian +2
We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i.e., $f(\mathbf{Z}, \mathbf{w}, \mathbf{a}) = \…
When is a Convolutional Filter Easy To Learn?
Simon S. Du, Jason D. Lee, Yuandong Tian
We analyze the convergence of (stochastic) gradient descent algorithm for learning a convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does n…
Channel-Recurrent Autoencoding for Image Modeling
Wenling Shang, Kihyuk Sohn, Yuandong Tian
Despite recent successes in synthesizing faces and bedrooms, existing generative models struggle to capture more complex image types, potentially due to the oversimplification of t…
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis
Yuandong Tian
In this paper, we explore theoretical properties of training a two-layered ReLU network with centered -dim…
Better Computer Go Player with Neural Network and Long-term Prediction
Yuandong Tian, Yan Zhu
Competing with top human players in the ancient game of Go has been a long-term goal of artificial intelligence. Go's high branching factor makes traditional search techniques inef…
Convolutional networks and learning invariant to homogeneous multiplicative scalings
Mark Tygert, Arthur Szlam, Soumith Chintala +3
The conventional classification schemes -- notably multinomial logistic regression -- used in conjunction with convolutional networks (convnets) are classical in statistics, design…