24 citations · 46 across the 9 of their papers we have counts for
9 papers · 1 filter
Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
Dylan Sam, Ayan Chakrabarti, Afshin Rostamizadeh +3
Measuring similarity between training examples is critical for curating high-quality and diverse pretraining datasets for language models. However, similarity is typically computed…
A Weighted K-Center Algorithm for Data Subset Selection
Srikumar Ramalingam, Pranjal Awasthi, Sanjiv Kumar
The success of deep learning hinges on enormous data and large models, which require labor-intensive annotations and heavy computation costs. Subset selection is a fundamental prob…
When does mixup promote local linearity in learned representations?
Arslan Chaudhry, Aditya Krishna Menon, Andreas Veit +3
Mixup is a regularization technique that artificially produces new samples using convex combinations of original training points. This simple technique has shown strong empirical p…
Balancing Robustness and Sensitivity using Feature Contrastive Learning
Seungyeon Kim, Daniel Glasner, Srikumar Ramalingam +3
It is generally believed that robust training of extremely large networks is critical to their success in real-world applications. However, when taken to the extreme, methods that…
Scaling Up Exact Neural Network Compression by ReLU Stability
Thiago Serra, Xin Yu, Abhinav Kumar +1
We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable. However, current a…
Kernelized Classification in Deep Networks
Sadeep Jayasumana, Srikumar Ramalingam, Sanjiv Kumar
We propose a kernelized classification layer for deep networks. Although conventional deep networks introduce an abundance of nonlinearity for representation (feature) learning, th…