19 citations · 37 across the 14 of their papers we have counts for
4 papers · 1 filter
How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?
Nikhil Garg, Jon Kleinberg, Kenny Peng
We introduce a mathematical framework for the linear representation hypothesis (LRH), which asserts that intermediate layers of language models store features linearly. We separate…
Position: Use Sparse Autoencoders to Discover Unknowns
Kenny Peng, Rajiv Movva, Jon Kleinberg +2
While sparse autoencoders (SAEs) have generated significant excitement, a series of negative results have added to skepticism about their usefulness. Here, we establish a conceptua…
REFORMS: Reporting Standards for Machine Learning Based Science
Sayash Kapoor, Emily Cantrell, Kenny Peng +16
Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and ge…
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Kenny Peng, Arunesh Mathur, Arvind Narayanan
Machine learning datasets have elicited concerns about privacy, bias, and unethical applications, leading to the retraction of prominent datasets such as DukeMTMC, MS-Celeb-1M, and…