GUIDE: Guided Initialization and Distillation of Embeddings

Khoa Trinh, Gaurav Menghani, Erik Vee

Algorithmic efficiency techniques such as distillation (\cite{hinton2015distillation}) are useful in improving model quality without increasing serving costs, provided a larger tea…