Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free

Ziyue Li, Tianyi Zhou

While large language models (LLMs) excel on generation tasks, their decoder-only architecture often limits their potential as embedding models if no further representation finetuni…