359 citations · 994 across the 28 of their papers we have counts for
28 papers
Accelerating Aggregation Queries on Unstructured Streams of Data
Matthew Russo, Tatsunori Hashimoto, Daniel Kang +2
Analysts and scientists are interested in querying streams of video, audio, and text to extract quantitative insights. For example, an urban planner may wish to measure congestion…
ZK-IMG: Attested Images via Zero-Knowledge Proofs to Fight Disinformation
Daniel Kang, Tatsunori Hashimoto, Ion Stoica +1
Over the past few years, AI methods of generating images have been increasing in capabilities, with recent breakthroughs enabling high-resolution, photorealistic "deepfakes" (artif…
Scaling up Trustless DNN Inference with Zero-Knowledge Proofs
Daniel Kang, Tatsunori Hashimoto, Ion Stoica +1
As ML models have increased in capabilities and accuracy, so has the complexity of their deployments. Increasingly, ML model consumers are turning to service providers to serve the…
Spurious Correlations in Reference-Free Evaluation of Text Generation
Esin Durmus, Faisal Ladhak, Tatsunori Hashimoto
Model-based, reference-free evaluation metrics have been proposed as a fast and cost-effective approach to evaluate Natural Language Generation (NLG) systems. Despite promising rec…
Extending the WILDS Benchmark for Unsupervised Adaptation
Shiori Sagawa, Pang Wei Koh, Tony Lee +17
Machine learning systems deployed in the wild are often trained on a source distribution but deployed on a different target distribution. Unlabeled data can be a powerful point of…
Large Language Models Can Be Strong Differentially Private Learners
Xuechen Li, Florian Tramèr, Percy Liang +1
Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stoch…
Proof: Accelerating Approximate Aggregation Queries with Expensive Predicates
Daniel Kang, John Guibas, Peter Bailis +3
Given a dataset , we are interested in computing the mean of a subset of which matches a predicate. ABae leverages stratified sampling and proxy models t…
Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions
Dorottya Demszky, Jing Liu, Zid Mancenido +4
In conversation, uptake happens when a speaker builds on the contribution of their interlocutor by, for example, acknowledging, repeating or reformulating what they have said. In e…
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
Tianyi Zhang, Tatsunori Hashimoto
We study how masking and predicting tokens in an unsupervised fashion can give rise to linguistic structures and downstream performance gains. Recent theories have suggested that p…
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal +53
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of auto…
Modeling the Second Player in Distributionally Robust Optimization
Paul Michel, Tatsunori Hashimoto, Graham Neubig
Distributionally robust optimization (DRO) provides a framework for training machine learning models that are able to perform well on a collection of related data distributions (th…
Robustness to Spurious Correlations via Human Annotations
Megha Srivastava, Tatsunori Hashimoto, Percy Liang
The reliability of machine learning systems critically assumes that the associations between features and labels remain similar between training and test distributions. However, un…
Distributionally Robust Losses for Latent Covariate Mixtures
John Duchi, Tatsunori Hashimoto, Hongseok Namkoong
While modern large-scale datasets often consist of heterogeneous subpopulations -- for example, multiple demographic groups or multiple text corpora -- the standard practice of min…
Improved Natural Language Generation via Loss Truncation
Daniel Kang, Tatsunori Hashimoto
Neural language models are usually trained to match the distributional properties of a large-scale corpus by minimizing the log loss. While straightforward to optimize, this approa…
Approximate Selection with Guarantees using Proxies
Daniel Kang, Edward Gan, Peter Bailis +2
Due to the falling costs of data acquisition and storage, researchers and industry analysts often want to find all instances of rare events in large datasets. For instance, scienti…
Learning Autocomplete Systems as a Communication Game
Mina Lee, Tatsunori B. Hashimoto, Percy Liang
We study textual autocomplete---the task of predicting a full sentence from a partial sentence---as a human-machine communication game. Specifically, we consider three competing go…
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto +1
Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlati…
Distributionally Robust Language Modeling
Yonatan Oren, Shiori Sagawa, Tatsunori B. Hashimoto +1
Language models are generally trained on data spanning a wide range of topics (e.g., news, reviews, fiction), but they might be applied to an a priori unknown target distribution (…
Unifying Human and Statistical Evaluation for Natural Language Generation
Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang
How can we measure whether a natural language generation system produces both high quality and diverse outputs? Human evaluation captures quality but not diversity, as it does not…
A Retrieve-and-Edit Framework for Predicting Structured Outputs
Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren +1
For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propos…
Fairness Without Demographics in Repeated Loss Minimization
Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong +1
Machine learning models (e.g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e.g., non-native speak…
Inferring Multidimensional Rates of Aging from Cross-Sectional Data
Emma Pierson, Pang Wei Koh, Tatsunori Hashimoto +4
Modeling how individuals evolve over time is a fundamental problem in the natural and social sciences. However, existing datasets are often cross-sectional with each individual obs…
Derivative free optimization via repeated classification
Tatsunori B. Hashimoto, Steve Yadlowsky, John C. Duchi
We develop an algorithm for minimizing a function using batched function value measurements at each of rounds by using classifiers to identify a function's sublevel set. We…
Unsupervised Transformation Learning via Convex Relaxations
Tatsunori B. Hashimoto, John C. Duchi, Percy Liang
Our goal is to extract meaningful transformations from raw images, such as varying the thickness of lines in handwriting or the lighting in a portrait. We propose an unsupervised a…