273 citations · 1.6k across the 53 of their papers we have counts for
40 papers · 1 filter
When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysis
Yiyou Sun, Zhenmei Shi, Yingyu Liang +1
Novel Class Discovery (NCD) aims at inferring novel classes in an unlabeled set by leveraging prior knowledge from a labeled set with known classes. Despite its importance, there i…
Stratified Adversarial Robustness with Rejection
Jiefeng Chen, Jayaram Raghuram, Jihye Choi +3
Recently, there is an emerging interest in adversarially training a classifier with a rejection option (also known as a selective classifier) for boosting adversarial robustness. W…
Domain Generalization via Nuclear Norm Regularization
Zhenmei Shi, Yifei Ming, Ying Fan +2
The ability to generalize to unseen domains is crucial for machine learning systems deployed in the real world, especially when we only have data from limited training domains. In…
The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning
Zhenmei Shi, Jiefeng Chen, Kunyang Li +4
Pre-training representations (a.k.a. foundation models) has recently become a prevalent learning paradigm, where one first pre-trains a representation using large-scale unlabeled d…
A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage over Fixed Features
Zhenmei Shi, Junyi Wei, Yingyu Liang
An important characteristic of neural networks is their ability to learn representations of the input data with effective features for prediction, which is believed to be a key fac…
On the identifiability of mixtures of ranking models
Xiaomin Zhang, Xucheng Zhang, Po-Ling Loh +1
Mixtures of ranking models are standard tools for ranking problems. However, even the fundamental question of parameter identifiability is not fully understood: the identifiability…
Towards Evaluating the Robustness of Neural Networks Learned by Transduction
Jiefeng Chen, Xi Wu, Yang Guo +2
There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021). Compare…
Attentive Walk-Aggregating Graph Neural Networks
Mehmet F. Demirel, Shengchao Liu, Siddhant Garg +2
Graph neural networks (GNNs) have been shown to possess strong representation power, which can be exploited for downstream prediction tasks on graph-structured data, such as molecu…
Towards Adversarial Robustness via Transductive Learning
Jiefeng Chen, Yang Guo, Xi Wu +4
There has been emerging interest to use transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020). Compared to traditional "test-time…
Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
Jiefeng Chen, Frederick Liu, Besim Avci +3
When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance…
Can Adversarial Weight Perturbations Inject Neural Backdoors?
Siddhant Garg, Adarsh Kumar, Vibhor Goel +1
Adversarial machine learning has exposed several security hazards of neural models and has become an important research topic in recent times. Thus far, the concept of an "adversar…
Functional Regularization for Representation Learning: A Unified Theoretical Perspective
Siddhant Garg, Yingyu Liang
Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks. While these approaches are widely used in…
Learning Entangled Single-Sample Gaussians in the Subset-of-Signals Model
Yingyu Liang, Hui Yuan
In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of distributions, given one single sample from each di…
ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
Jiefeng Chen, Yixuan Li, Xi Wu +2
Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions can be brittle…
Gradients as Features for Deep Representation Learning
Fangzhou Mu, Yingyu Liang, Yin Li
We address the challenging problem of deep representation learning--the efficient adaption of a pre-trained deep network to different tasks. Specifically, we propose to explore gra…
Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation
Xi Wu, Yang Guo, Jiefeng Chen +3
We consider representation learning (hypothesis class ) where training and test distributions can be different. Recent studies provide hi…
Learning Entangled Single-Sample Distributions via Iterative Trimming
Hui Yuan, Yingyu Liang
In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of distributions, given one \emph{single} sample from each…
Robust Out-of-distribution Detection for Neural Networks
Jiefeng Chen, Yixuan Li, Xi Wu +2
Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in the real world. Existing approaches for detecting OOD examples work well when ev…
Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis
Zhongkai Sun, Prathusha Sarma, William Sethares +1
Multimodal language analysis often considers relationships between features based on text and those based on acoustical and visual properties. Text features typically outperform no…
Robust Attribution Regularization
Jiefeng Chen, Xi Wu, Vaibhav Rastogi +2
An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions. We take a step towards solving this problem throug…
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
Zeyuan Allen-Zhu, Yuanzhi Li, Yingyu Liang
The fundamental learning theory behind neural networks remains largely open. What classes of functions can neural networks actually learn? Why doesn't the trained network overfit w…
Recovery Guarantees for Quadratic Tensors with Sparse Observations
Hongyang R. Zhang, Vatsal Sharan, Moses Charikar +1
We consider the tensor completion problem of predicting the missing entries of a tensor. The commonly used CP model has a triple product form, but an alternate family of quadratic…
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
Yuanzhi Li, Yingyu Liang
Neural networks have many successful applications, while much less theoretical understanding has been gained. Towards bridging this gap, we study the problem of learning a two-laye…
N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules
Shengchao Liu, Mehmet Furkan Demirel, Yingyu Liang
Machine learning techniques have recently been adopted in various applications in medicine, biology, chemistry, and material engineering. An important task is to predict the proper…