31 citations · 180 across the 32 of their papers we have counts for
4 papers · 1 filter
Time is Encoded in the Weights of Finetuned Language Models
Kai Nylund, Suchin Gururangan, Noah A. Smith
We present time vectors, a simple tool to customize language models to new time periods. Time vectors are created by finetuning a language model on data from a single time (e.g., a…
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore
Sewon Min, Suchin Gururangan, Eric Wallace +4
The legality of training language models (LMs) on copyrighted or otherwise restricted data is under intense debate. However, as we show, model performance significantly degrades if…
Information Flow Control in Machine Learning through Modular Model Architecture
Trishita Tiwari, Suchin Gururangan, Chuan Guo +7
In today's machine learning (ML) models, any part of the training data can affect the model output. This lack of control for information flow from training data to model output is…
Scaling Expert Language Models with Unsupervised Domain Discovery
Suchin Gururangan, Margaret Li, Mike Lewis +4
Large language models are typically trained densely: all parameters are updated with respect to all inputs. This requires synchronization of billions of parameters across thousands…