288 citations · 2.1k across the 42 of their papers we have counts for
42 papers
Understanding Robustness in Teacher-Student Setting: A New Perspective
Zhuolin Yang, Zhaoxi Chen, Tiffany Cai +3
Adversarial examples have appeared as a ubiquitous property of machine learning models where bounded adversarial perturbation could mislead the models to make arbitrarily incorrect…
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Yuandong Tian, Xinlei Chen, Surya Ganguli
While contrastive approaches of self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs)…
Multi-Agent Collaboration via Reward Attribution Decomposition
Tianjun Zhang, Huazhe Xu, Xiaolong Wang +4
Recent advances in multi-agent reinforcement learning (MARL) have achieved super-human performance in games like Quake 3 and Dota 2. Unfortunately, these techniques require orders-…
Understanding Self-supervised Learning with Dual Deep Networks
Yuandong Tian, Lantao Yu, Xinlei Chen +1
We propose a novel theoretical framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks (e.g., SimCLR). First, we pro…
Real-world Video Adaptation with Reinforcement Learning
Hongzi Mao, Shannon Chen, Drew Dimmery +5
Client-side video players employ adaptive bitrate (ABR) algorithms to optimize user quality of experience (QoE). We evaluate recently proposed RL-based ABR methods in Facebook's we…
Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion
Tianyu Li, Roberto Calandra, Deepak Pathak +3
Hierarchical learning has been successful at learning generalizable locomotion skills on walking robots in a sample-efficient manner. However, the low-dimensional "latent" action u…
Joint Policy Search for Multi-agent Collaboration with Imperfect Information
Yuandong Tian, Qucheng Gong, Tina Jiang
To learn good joint policies for multi-agent collaboration with imperfect information remains a fundamental challenge. While for two-player zero-sum games, coordinate-ascent approa…
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
Linnan Wang, Rodrigo Fonseca, Yuandong Tian
High dimensional black-box optimization has broad applications but remains a challenging problem to solve. Given a set of samples $\{\vx_i, y_i\}$, building a global model (like Ba…
Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
Qingquan Song, Dehua Cheng, Hanning Zhou +3
Click-Through Rate (CTR) prediction is one of the most important machine learning tasks in recommender systems, driving personalized experience for billions of consumers. Neural ar…
Few-shot Neural Architecture Search
Yiyang Zhao, Linnan Wang, Yuandong Tian +2
Efficient evaluation of a network architecture drawn from a large search space remains a key challenge in Neural Architecture Search (NAS). Vanilla NAS evaluates each architecture…
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
Xiaoliang Dai, Alvin Wan, Peizhao Zhang +8
Neural Architecture Search (NAS) yields state-of-the-art neural networks that outperform their best manually-designed counterparts. However, previous NAS methods search for archite…
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
Alvin Wan, Xiaoliang Dai, Peizhao Zhang +9
Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space i…
Bayesian Relational Memory for Semantic Visual Navigation
Yi Wu, Yuxin Wu, Aviv Tamar +3
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where…
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension
Yuandong Tian
We consider a deep ReLU / Leaky ReLU student network trained from the output of a fixed teacher network of the same depth, with Stochastic Gradient Descent (SGD). The student netwo…
A Neural-based Program Decompiler
Cheng Fu, Huili Chen, Haolan Liu +4
Reverse engineering of binary executables is a critical problem in the computer security domain. On the one hand, malicious parties may recover interpretable source codes from the…
Luck Matters: Understanding Training Dynamics of Deep ReLU Networks
Yuandong Tian, Tina Jiang, Qucheng Gong +1
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship betw…
Sample-Efficient Neural Architecture Search by Learning Action Space
Linnan Wang, Saining Xie, Teng Li +2
Neural Architecture Search (NAS) has emerged as a promising technique for automatic neural network design. However, existing MCTS based NAS approaches often utilize manually design…
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
Ari S. Morcos, Haonan Yu, Michela Paganini +1
The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately…
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
Haonan Yu, Sergey Edunov, Yuandong Tian +1
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization…
Hierarchical Decision Making by Generating and Following Natural Language Instructions
Hengyuan Hu, Denis Yarats, Qucheng Gong +2
We explore using latent natural language instructions as an expressive and compositional representation of complex actions for hierarchical decision making. Rather than directly se…
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero
Yuandong Tian, Jerry Ma, Qucheng Gong +4
The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the c…
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Bichen Wu, Xiaoliang Dai, Peizhao Zhang +7
Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS…
Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
Bichen Wu, Yanghan Wang, Peizhao Zhang +3
Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices wi…
Learning to Perform Local Rewriting for Combinatorial Optimization
Xinyun Chen, Yuandong Tian
Search-based methods for hard combinatorial optimization are often guided by heuristics. Tuning heuristics in various conditions and situations is often time-consuming. In this pap…