Large Language Models Are Overparameterized Text Encoders

Thennal D K, Tim Fischer, Chris Biemann

Large language models (LLMs) demonstrate strong performance as text embedding models when finetuned with supervised contrastive training. However, their large size balloons inferen…