6 citations · 11 across the 6 of their papers we have counts for
7 papers
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Jacqueline He, Howard Yen, Shuyue Stella Li +9
Logit-based knowledge distillation (KD) is used to train smaller language models (LMs) via supervision from stronger teachers, but whether its benefits are consistent across traini…
Precise Information Control in Long-Form Text Generation
Jacqueline He, Howard Yen, Margaret Li +7
A central challenge in language models (LMs) is faithfulness hallucination: the generation of information unsubstantiated by input context. To study this problem, we propose Precis…
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
Akari Asai, Jacqueline He, Rulin Shao +22
Scientific progress depends on researchers' ability to synthesize the growing body of literature. Can large language models (LMs) assist scientists in this task? We introduce OpenS…
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
Rulin Shao, Jacqueline He, Akari Asai +5
Scaling laws with respect to the amount of training data and the number of parameters allow us to predict the cost-benefit trade-offs of pretraining language models (LMs) in differ…
Challenges in Context-Aware Neural Machine Translation
Linghao Jin, Jacqueline He, Jonathan May +1
Context-aware neural machine translation involves leveraging information beyond sentence-level context to resolve inter-sentential discourse dependencies and improve document-level…
MABEL: Attenuating Gender Bias using Textual Entailment Data
Jacqueline He, Mengzhou Xia, Christiane Fellbaum +1
Pre-trained language models encode undesirable social biases, which are further exacerbated in downstream use. To this end, we propose MABEL (a Method for Attenuating Gender Bias u…
Can Rationalization Improve Robustness?
Howard Chen, Jacqueline He, Karthik Narasimhan +1
A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper,…