2.3k citations · 11.9k across the 167 of their papers we have counts for
22 papers · 1 filter
Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss
Stephen Mussmann, Percy Liang
Uncertainty sampling, a popular active learning algorithm, is used to reduce the amount of data required to learn a classifier, but it has been observed in practice to converge to…
A Retrieve-and-Edit Framework for Predicting Structured Outputs
Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren +1
For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propos…
FrAngel: Component-Based Synthesis with Control Structures
Kensen Shi, Jacob Steinhardt, Percy Liang
In component-based program synthesis, the synthesizer generates a program given a library of components (functions). Existing component-based synthesizers have difficulty synthesiz…
Semidefinite relaxations for certifying robustness to adversarial examples
Aditi Raghunathan, Jacob Steinhardt, Percy Liang
Despite their impressive performance on diverse tasks, neural networks fail catastrophically in the presence of adversarial inputs---imperceptibly but adversarially perturbed versi…
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
Pang Wei Koh, Jacob Steinhardt, Percy Liang
Machine learning models trained on data from the outside world can be corrupted by data poisoning attacks that inject malicious points into the models' training sets. A common defe…
Mapping Natural Language Commands to Web Elements
Panupong Pasupat, Tian-Shun Jiang, Evan Zheran Liu +2
The web provides a rich, open-domain environment with textual, structural, and spatial properties. We propose a new task for grounding language in this environment: given a natural…
Transforming Question Answering Datasets Into Natural Language Inference Datasets
Dorottya Demszky, Kelvin Guu, Percy Liang
Existing datasets for natural language inference (NLI) have propelled research on language understanding. We propose a new method for automatically deriving NLI datasets from the g…
Textual Analogy Parsing: What's Shared and What's Compared among Analogous Facts
Matthew Lamm, Arun Tejasvi Chaganty, Christopher D. Manning +2
To understand a sentence like "whereas only 10% of White Americans live at or below the poverty line, 28% of African Americans do" it is important not only to identify individual f…
Decoupling Strategy and Generation in Negotiation Dialogues
He He, Derek Chen, Anusha Balakrishnan +1
We consider negotiation settings in which two agents use natural language to bargain on goods. Agents need to decide on both high-level strategy (e.g., proposing $50) and the exec…
QuAC : Question Answering in Context
Eunsol Choi, He He, Mohit Iyyer +5
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1)…
Fairness Without Demographics in Repeated Loss Minimization
Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong +1
Machine learning models (e.g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e.g., non-native speak…
The price of debiasing automatic metrics in natural language evaluation
Arun Tejasvi Chaganty, Stephen Mussman, Percy Liang
For evaluating generation systems, automatic metrics such as BLEU cost nothing to run but have been shown to correlate poorly with human judgment, leading to systematic bias agains…
Inferring Multidimensional Rates of Aging from Cross-Sectional Data
Emma Pierson, Pang Wei Koh, Tatsunori Hashimoto +4
Modeling how individuals evolve over time is a fundamental problem in the natural and social sciences. However, existing datasets are often cross-sectional with each individual obs…
On the Relationship between Data Efficiency and Error for Uncertainty Sampling
Stephen Mussmann, Percy Liang
While active learning offers potential cost savings, the actual data efficiency---the reduction in amount of labeled data needed to obtain the same error rate---observed in practic…
Know What You Don't Know: Unanswerable Questions for SQuAD
Pranav Rajpurkar, Robin Jia, Percy Liang
Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for whi…
Planning, Inference and Pragmatics in Sequential Language Games
Fereshte Khani, Noah D. Goodman, Percy Liang
We study sequential language games in which two players, each with private information, communicate to achieve a common goal. In such games, a successful player must (i) infer the…
Training Classifiers with Natural Language Explanations
Braden Hancock, Paroma Varma, Stephanie Wang +3
Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a f…
Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
Juncen Li, Robin Jia, He He +1
We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., c…
Generalized Binary Search For Split-Neighborly Problems
Stephen Mussmann, Percy Liang
In sequential hypothesis testing, Generalized Binary Search (GBS) greedily chooses the test with the highest information gain at each step. It is known that GBS obtains the gold st…
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat +2
Reinforcement learning (RL) agents improve through trial-and-error, but when reward is sparse and the agent cannot discover successful action sequences, learning stagnates. This ha…
Learning a SAT Solver from Single-Bit Supervision
Daniel Selsam, Matthew Lamm, Benedikt Bünz +3
We present NeuroSAT, a message passing neural network that learns to solve SAT problems after only being trained as a classifier to predict satisfiability. Although it is not compe…
Certified Defenses against Adversarial Examples
Aditi Raghunathan, Jacob Steinhardt, Percy Liang
While neural networks have achieved high accuracy on standard image classification benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbation…