1.2k citations · 1.2k across the 4 of their papers we have counts for
4 papers
Large Language Models Can Self-Improve
Jiaxin Huang, Shixiang Shane Gu, Le Hou +4
Large Language Models (LLMs) have achieved excellent performances in various tasks. However, fine-tuning an LLM requires extensive supervision. Human, on the other hand, may improv…
Scaling Instruction-Finetuned Language Models
Hyung Won Chung, Le Hou, Shayne Longpre +32
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we expl…
Token Dropping for Efficient BERT Pretraining
Le Hou, Richard Yuanzhe Pang, Tianyi Zhou +4
Transformer-based models generally allocate the same amount of computation for each token in a given sequence. We develop a simple but effective "token dropping" method to accelera…
Speeding up Deep Model Training by Sharing Weights and Then Unsharing
Shuo Yang, Le Hou, Xiaodan Song +2
We propose a simple and efficient approach for training the BERT model. Our approach exploits the special structure of BERT that contains a stack of repeated modules (i.e., transfo…