2.4k citations · 3k across the 56 of their papers we have counts for
6 papers · 2 filters
Copy-Paste Language Models
Nathan Godey, Yoav Artzi
Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can be improved through explicit…
Co-Evolving Structured Knowledge and Reasoning in Language Models
Ryan Thomas Noonan, Linxi Zhao, Menghan Xu +6
Retrieval-augmented methods improve factual accuracy by grounding language models in external knowledge, but retrieving over unstructured text often introduces irrelevant context a…
CT: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees
S M Rafiuddin, Atriya Sen
Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a branching reply tree. We study…
Co-LMLM: Continuous-Query Limited Memory Language Models
Yair Feldman, Linxi Zhao, Nathan Godey +5
Externalizing knowledge in LLM pre-training is a promising avenue to achieve higher performance at smaller scales, control knowledge use, and overall increase model transparency. W…
The State-Prediction Separation Hypothesis
Giovanni Monea, Nathan Godey, Kianté Brantley +1
Transformers use the same forward computation stream to both predict the next token and store useful state for future token predictions. We formulate the \emph{state-prediction sep…
Lost in Backpropagation: The LM Head is a Gradient Bottleneck
Nathan Godey, Yoav Artzi
The last layer of neural language models (LMs) projects output features of dimension to logits in dimension , the size of the vocabulary, where usually . This misma…