15 citations · 54 across the 13 of their papers we have counts for
13 papers
Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models
Mayee F. Chen, Nicholas Roberts, Kush Bhatia +4
The quality of training data impacts the performance of pre-trained large language models (LMs). Given a fixed budget of tokens, we study how to best select data that leads to good…
Geometry-Aware Adaptation for Pretrained Models
Nicholas Roberts, Xintong Li, Dyah Adila +4
Machine learning models -- including prominent zero-shot models -- are often trained on datasets whose labels are only a small proportion of a larger label space. Such spaces are c…
Lifting Weak Supervision To Structured Prediction
Harit Vishwakarma, Nicholas Roberts, Frederic Sala
Weak supervision (WS) is a rich set of techniques that produce pseudolabels by aggregating easily obtained but potentially noisy label estimates from a variety of sources. WS is th…
AutoML for Climate Change: A Call to Action
Renbo Tu, Nicholas Roberts, Vishak Prasad +7
The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications. The climate c…
AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels
Nicholas Roberts, Xintong Li, Tzu-Heng Huang +7
Weak supervision (WS) is a powerful method to build labeled datasets for training supervised models in the face of little-to-no labeled data. It replaces hand-labeling data with ag…
Generative Modeling Helps Weak Supervision (and Vice Versa)
Benedikt Boecking, Nicholas Roberts, Willie Neiswanger +3
Many promising applications of supervised machine learning face hurdles in the acquisition of labeled data in sufficient quantity and quality, creating an expensive bottleneck. To…
Universalizing Weak Supervision
Changho Shin, Winfred Li, Harit Vishwakarma +2
Weak supervision (WS) frameworks are a popular way to bypass hand-labeling large datasets for training data-hungry models. These approaches synthesize multiple noisy but cheaply-ac…
NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks
Renbo Tu, Nicholas Roberts, Mikhail Khodak +3
Most existing neural architecture search (NAS) benchmarks and algorithms prioritize well-studied tasks, e.g. image classification on CIFAR or ImageNet. This makes the performance o…
Mining logical circuits in fungi
Nic Roberts, Andrew Adamatzky
Living substrates are capable for nontrivial mappings of electrical signals due to the substrate nonlinear electrical characteristics. This property can be used to realise Boolean…
Rethinking Neural Operations for Diverse Tasks
Nicholas Roberts, Mikhail Khodak, Tri Dao +3
An important goal of AutoML is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this goal, we study the problem of enabling users…
Decoding and Diversity in Machine Translation
Nicholas Roberts, Davis Liang, Graham Neubig +1
Neural Machine Translation (NMT) systems are typically evaluated using automated metrics that assess the agreement between generated translations and ground truth candidates. To im…
Model Weight Theft With Just Noise Inputs: The Curious Case of the Petulant Attacker
Nicholas Roberts, Vinay Uday Prabhu, Matthew McAteer
This paper explores the scenarios under which an attacker can claim that 'Noise and access to the softmax layer of the model is all you need' to steal the weights of a convolutiona…
Grassmannian Packings in Neural Networks: Learning with Maximal Subspace Packings for Diversity and Anti-Sparsity
Dian Ang Yap, Nicholas Roberts, Vinay Uday Prabhu
Kernel sparsity ("dying ReLUs") and lack of diversity are commonly observed in CNN kernels, which decreases model capacity. Drawing inspiration from information theory and wireless…