101 citations · 269 across the 17 of their papers we have counts for
17 papers
Understanding Finetuning for Factual Knowledge Extraction
Gaurav Ghosal, Tatsunori Hashimoto, Aditi Raghunathan
In this work, we study the impact of QA fine-tuning data on downstream factuality. We show that fine-tuning on lesser-known facts that are poorly stored during pretraining yields s…
Trustless Audits without Revealing Data or Models
Suppakit Waiwitlikhit, Ion Stoica, Yi Sun +2
There is an increasing conflict between business incentives to hide models and data as trade secrets, and the societal need for algorithmic transparency. For example, a rightsholde…
Language Models with Conformal Factuality Guarantees
Christopher Mohri, Tatsunori Hashimoto
Guaranteeing the correctness and factuality of language model (LM) outputs is a major open problem. In this work, we propose conformal factuality, a framework that can ensure high…
MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks
Allen Nie, Yuhui Zhang, Atharva Amdekar +3
Human commonsense understanding of the physical and social world is organized around intuitive theories. These theories support making causal and moral judgments. When something ba…
On the Fairness ROAD: Robust Optimization for Adversarial Debiasing
Vincent Grari, Thibault Laugel, Tatsunori Hashimoto +2
In the field of algorithmic fairness, significant attention has been put on group fairness criteria, such as Demographic Parity and Equalized Odds. Nevertheless, these objectives,…
Benchmarking and Improving Generator-Validator Consistency of Language Models
Xiang Lisa Li, Vaishnavi Shrivastava, Siyan Li +2
As of September 2023, ChatGPT correctly answers "what is 7+8" with 15, but when asked "7+8=15, True or False" it responds with "False". This inconsistency between generating and va…