49 citations · 81 across the 12 of their papers we have counts for
12 papers
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Howard Chen, Noam Razin, Karthik Narasimhan +1
Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting. In t…
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Howard Chen, Ramakanth Pasunuru, Jason Weston +1
Large language models (LLMs) have advanced in large strides due to the effectiveness of the self-attention mechanism that processes and compares all tokens at once. However, this m…
COLLIE: Systematic Construction of Constrained Text Generation Tasks
Shunyu Yao, Howard Chen, Austin W. Hanjie +2
Text generation under constraints have seen increasing interests in natural language processing, especially with the rapidly improving capabilities of large language models. Howeve…
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning
Jane Pan, Tianyu Gao, Howard Chen +1
Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood. Some works suggest tha…
C-STS: Conditional Semantic Textual Similarity
Ameet Deshpande, Carlos E. Jimenez, Howard Chen +6
Semantic textual similarity (STS), a cornerstone task in NLP, measures the degree of similarity between a pair of sentences, and has broad application in fields such as information…
Controllable Text Generation with Language Constraints
Howard Chen, Huihan Li, Danqi Chen +1
We consider the task of text generation in language models with constraints specified in natural language. To this end, we first create a challenging benchmark Cognac that provides…