1.4k citations · 1.6k across the 13 of their papers we have counts for
8 papers · 1 filter
Scalable Influence and Fact Tracing for Large Language Model Pretraining
Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi +2
Training data attribution (TDA) methods aim to attribute model outputs back to specific training examples, and the application of these methods to large language model (LLM) output…
Self-Influence Guided Data Reweighting for Language Model Pre-training
Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy +3
Language Models (LMs) pre-trained with self-supervision on large text corpora have become the default starting point for developing models for various NLP tasks. Once the pre-train…
Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning
Maximilian Mozes, Tolga Bolukbasi, Ann Yuan +3
Pretrained large language models (LLMs) are able to solve a wide variety of tasks through transfer learning. Various explainability methods have been developed to investigate their…
Towards Agile Text Classifiers for Everyone
Maximilian Mozes, Jessica Hoffmann, Katrin Tomanek +5
Text-based safety classifiers are widely used for content moderation and increasingly to tune generative language model behavior - a topic of growing concern for the safety of digi…
Towards Tracing Factual Knowledge in Language Models Back to the Training Data
Ekin Akyürek, Tolga Bolukbasi, Frederick Liu +4
Language models (LMs) have been shown to memorize a great deal of factual knowledge contained in their training data. But when an LM generates an assertion, it is often difficult t…
An Interpretability Illusion for BERT
Tolga Bolukbasi, Adam Pearce, Ann Yuan +4
We describe an "interpretability illusion" that arises when analyzing the BERT model. Activations of individual neurons in the network may spuriously appear to encode a single, sim…
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Zou +2
The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text…
Quantifying and Reducing Stereotypes in Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Zou +2
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the o…