68 citations · 137 across the 8 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 1 cited
Teacher Guided Training: An Efficient Framework for Knowledge Transfer
Manzil Zaheer, Ankit Singh Rawat, Seungyeon Kim +5
The remarkable performance gains realized by large pretrained models, e.g., GPT-3, hinge on the massive amounts of data they are exposed to during training. Analogously, distilling…
cs.CL2022
Treeformer: Dense Gradient Trees for Efficient Attention Computation
Lovish Madaan, Srinadh Bhojanapalli, Himanshu Jain +1
Standard inference and training with transformer based architectures scale quadratically with input sequence length. This is prohibitively large for a variety of applications espec…