273 citations · 1.6k across the 53 of their papers we have counts for
56 papers
When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysis
Yiyou Sun, Zhenmei Shi, Yingyu Liang +1
Novel Class Discovery (NCD) aims at inferring novel classes in an unlabeled set by leveraging prior knowledge from a labeled set with known classes. Despite its importance, there i…
Stratified Adversarial Robustness with Rejection
Jiefeng Chen, Jayaram Raghuram, Jihye Choi +3
Recently, there is an emerging interest in adversarially training a classifier with a rejection option (also known as a selective classifier) for boosting adversarial robustness. W…
Domain Generalization via Nuclear Norm Regularization
Zhenmei Shi, Yifei Ming, Ying Fan +2
The ability to generalize to unseen domains is crucial for machine learning systems deployed in the real world, especially when we only have data from limited training domains. In…
The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning
Zhenmei Shi, Jiefeng Chen, Kunyang Li +4
Pre-training representations (a.k.a. foundation models) has recently become a prevalent learning paradigm, where one first pre-trains a representation using large-scale unlabeled d…
A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage over Fixed Features
Zhenmei Shi, Junyi Wei, Yingyu Liang
An important characteristic of neural networks is their ability to learn representations of the input data with effective features for prediction, which is believed to be a key fac…
What Knowledge Gets Distilled in Knowledge Distillation?
Utkarsh Ojha, Yuheng Li, Anirudh Sundara Rajan +2
Knowledge distillation aims to transfer useful information from a teacher network to a student network, with the primary goal of improving the student's performance for the task at…
On the identifiability of mixtures of ranking models
Xiaomin Zhang, Xucheng Zhang, Po-Ling Loh +1
Mixtures of ranking models are standard tools for ranking problems. However, even the fundamental question of parameter identifiability is not fully understood: the identifiability…
Towards Evaluating the Robustness of Neural Networks Learned by Transduction
Jiefeng Chen, Xi Wu, Yang Guo +2
There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021). Compare…
Attentive Walk-Aggregating Graph Neural Networks
Mehmet F. Demirel, Shengchao Liu, Siddhant Garg +2
Graph neural networks (GNNs) have been shown to possess strong representation power, which can be exploited for downstream prediction tasks on graph-structured data, such as molecu…
Towards Adversarial Robustness via Transductive Learning
Jiefeng Chen, Yang Guo, Xi Wu +4
There has been emerging interest to use transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020). Compared to traditional "test-time…
Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
Jiefeng Chen, Frederick Liu, Besim Avci +3
When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance…
Deep Online Fused Video Stabilization
Zhenmei Shi, Fuhao Shi, Wei-Sheng Lai +2
We present a deep neural network (DNN) that uses both sensor data (gyroscope) and image content (optical flow) to stabilize videos through unsupervised learning. The network fuses…
PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding
Zhao Jinman, Shawn Zhong, Xiaomin Zhang +1
We look into the task of \emph{generalizing} word embeddings: given a set of pre-trained word vectors over a finite vocabulary, the goal is to predict embedding vectors for out-of-…
Graph Neural Networks for an Accurate and Interpretable Prediction of the Properties of Polycrystalline Materials
Minyi Dai, Mehmet F. Demirel, Yingyu Liang +1
Various machine learning models have been used to predict the properties of polycrystalline materials, but none of them directly consider the physical interactions among neighborin…
Can Adversarial Weight Perturbations Inject Neural Backdoors?
Siddhant Garg, Adarsh Kumar, Vibhor Goel +1
Adversarial machine learning has exposed several security hazards of neural models and has become an important research topic in recent times. Thus far, the concept of an "adversar…
Functional Regularization for Representation Learning: A Unified Theoretical Perspective
Siddhant Garg, Yingyu Liang
Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks. While these approaches are widely used in…
Learning Entangled Single-Sample Gaussians in the Subset-of-Signals Model
Yingyu Liang, Hui Yuan
In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of distributions, given one single sample from each di…
ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
Jiefeng Chen, Yixuan Li, Xi Wu +2
Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions can be brittle…
Gradients as Features for Deep Representation Learning
Fangzhou Mu, Yingyu Liang, Yin Li
We address the challenging problem of deep representation learning--the efficient adaption of a pre-trained deep network to different tasks. Specifically, we propose to explore gra…
Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation
Xi Wu, Yang Guo, Jiefeng Chen +3
We consider representation learning (hypothesis class ) where training and test distributions can be different. Recent studies provide hi…
Learning Entangled Single-Sample Distributions via Iterative Trimming
Hui Yuan, Yingyu Liang
In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of distributions, given one \emph{single} sample from each…
Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer
Siddhant Garg, Rohit Kumar Sharma, Yingyu Liang
Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. In this paper we show that concatenating the embeddings from the pre-tr…
Robust Out-of-distribution Detection for Neural Networks
Jiefeng Chen, Yixuan Li, Xi Wu +2
Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in the real world. Existing approaches for detecting OOD examples work well when ev…
Sketching Transformed Matrices with Applications to Natural Language Processing
Yingyu Liang, Zhao Song, Mengdi Wang +2
Suppose we are given a large matrix that cannot be stored in memory but is in a disk or is presented in a data stream. However, we need to compute a matrix decomposit…