Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?

Yixuan Tang, Yi Yang

The significant advancements of Large Language Models (LLMs) in generative tasks have led to a growing body of work exploring LLM-based embedding models. While these models, employ…