47 citations · 87 across the 6 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024★ 1 cited
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh +12
A primary challenge in large language model (LLM) development is their onerous pre-training cost. Typically, such pre-training involves optimizing a self-supervised objective (such…
cs.LG2023★ 4 cited
EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval
Seungyeon Kim, Ankit Singh Rawat, Manzil Zaheer +6
Large neural models (such as Transformers) achieve state-of-the-art performance for information retrieval (IR). In this paper, we aim to improve distillation methods that pave the…