278 citations · 618 across the 32 of their papers we have counts for
23 papers · 1 filter
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
Ke Ye, Heinrich Jiang, Afshin Rostamizadeh +6
Pre-training large language models is known to be extremely resource intensive and often times inefficient, under-utilizing the information encapsulated in the training text sequen…
Is margin all you need? An extensive empirical study of active learning on tabular data
Dara Bahri, Heinrich Jiang, Tal Schuster +1
Given a labeled training set and a collection of unlabeled data, the goal of active learning (AL) is to identify the best unlabeled points to label. In this comprehensive study, we…
Predicting on the Edge: Identifying Where a Larger Model Does Better
Taman Narayan, Heinrich Jiang, Sen Zhao +1
Much effort has been devoted to making large and more accurate models, but relatively little has been put into understanding which examples are benefiting from the added complexity…
Active Covering
Heinrich Jiang, Afshin Rostamizadeh
We analyze the problem of active covering, where the learner is given an unlabeled dataset and can sequentially label query examples. The objective is to label query all of the pos…
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
Dara Bahri, Heinrich Jiang, Yi Tay +1
Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders o…
Churn Reduction via Distillation
Heinrich Jiang, Harikrishna Narasimhan, Dara Bahri +2
In real-world systems, models are frequently updated as more data becomes available, and in addition to achieving high accuracy, the goal is to also maintain a low difference in pr…
Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection
Dara Bahri, Heinrich Jiang, Yi Tay +1
Detecting out-of-distribution (OOD) examples is critical in many applications. We propose an unsupervised method to detect OOD samples using a -NN density estimate with respect…
Locally Adaptive Label Smoothing for Predictive Churn
Dara Bahri, Heinrich Jiang
Training modern neural networks is an inherently noisy process that can lead to high \emph{prediction churn} -- disagreements between re-trainings of the same model due to factors…
Stochastic Bandits with Linear Constraints
Aldo Pacchiano, Mohammad Ghavamzadeh, Peter Bartlett +1
We study a constrained contextual linear bandit setting, where the goal of the agent is to produce a sequence of policies, whose expected cumulative reward over the course of r…
Faster DBSCAN via subsampled similarity queries
Heinrich Jiang, Jennifer Jang, Jakub Łącki
DBSCAN is a popular density-based clustering algorithm. It computes the -neighborhood graph of a dataset and uses the connected components of the high-degree nodes to decide the…
Learning the Truth From Only One Side of the Story
Heinrich Jiang, Qijia Jiang, Aldo Pacchiano
Learning under one-sided feedback (i.e., where we only observe the labels for examples we predicted positively on) is a fundamental problem in machine learning -- applications incl…
Deep k-NN for Noisy Labels
Dara Bahri, Heinrich Jiang, Maya Gupta
Modern machine learning models are often trained on examples with noisy labels that hurt performance and are hard to identify. In this paper, we provide an empirical study showing…
Robustness Guarantees for Mode Estimation with an Application to Bandits
Aldo Pacchiano, Heinrich Jiang, Michael I. Jordan
Mode estimation is a classical problem in statistics with a wide range of applications in machine learning. Despite this, there is little understanding in its robustness properties…
Group-based Fair Learning Leads to Counter-intuitive Predictions
Ofir Nachum, Heinrich Jiang
A number of machine learning (ML) methods have been proposed recently to maximize model predictive accuracy while enforcing notions of group parity or fairness across sub-populatio…
Minimum-Margin Active Learning
Heinrich Jiang, Maya Gupta
We present a new active sampling method we call min-margin which trains multiple learners on bootstrap samples and then chooses the examples to label based on the candidates' minim…
Identifying and Correcting Label Bias in Machine Learning
Heinrich Jiang, Ofir Nachum
Datasets often contain biases which unfairly disadvantage certain groups, and classifiers trained on such datasets can inherit these biases. In this paper, we provide a mathematica…
DBSCAN++: Towards fast and scalable density clustering
Jennifer Jang, Heinrich Jiang
DBSCAN is a classical density-based clustering procedure with tremendous practical relevance. However, DBSCAN implicitly needs to compute the empirical density for each sample poin…
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
Andrew Cotter, Heinrich Jiang, Serena Wang +4
We show that many machine learning goals, such as improved fairness metrics, can be expressed as constraints on the model's predictions, which we call rate constraints. We study th…
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
Andrew Cotter, Maya Gupta, Heinrich Jiang +5
Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals. We study the gene…
Interpretable Set Functions
Andrew Cotter, Maya Gupta, Heinrich Jiang +4
We propose learning flexible but interpretable functions that aggregate a variable-length set of permutation-invariant feature vectors to predict a label. We use a deep lattice net…
Quickshift++: Provably Good Initializations for Sample-Based Mean Shift
Heinrich Jiang, Jennifer Jang, Samory Kpotufe
We provide initial seedings to the Quick Shift clustering algorithm, which approximate the locally high-density regions of the data. Such seedings act as more stable and expressive…
Two-Player Games for Efficient Non-Convex Constrained Optimization
Andrew Cotter, Heinrich Jiang, Karthik Sridharan
In recent years, constrained optimization has become increasingly relevant to the machine learning community, with applications including Neyman-Pearson classification, robust opti…
Nonparametric Stochastic Contextual Bandits
Melody Y. Guan, Heinrich Jiang
We analyze the -armed bandit problem where the reward for each arm is a noisy realization based on an observed context under mild nonparametric assumptions. We attain tight resu…