4 citations · 15 across the 5 of their papers we have counts for
5 papers · 1 filter
How much do language models memorize?
John X. Morris, Chawin Sitawarin, Chuan Guo +5
We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memori…
Contextual Document Embeddings
John X. Morris, Alexander M. Rush
Dense document embeddings are central to neural retrieval. The dominant paradigm is to train and construct embeddings by running encoders directly on individual documents. In this…
Language Model Inversion
John X. Morris, Wenting Zhao, Justin T. Chiu +2
Language models produce a distribution over the next token; can we use this information to recover the prompt tokens? We consider the problem of language model inversion and show t…
Tree Prompting: Efficient Task Adaptation without Fine-Tuning
John X. Morris, Chandan Singh, Alexander M. Rush +2
Prompting language models (LMs) is the main interface for applying them to new tasks. However, for smaller LMs, prompting provides low accuracy compared to gradient-based finetunin…
Text Embeddings Reveal (Almost) As Much As Text
John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov +1
How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented…