4 papers
ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs
Ailiang Lin, Zhuoyun Li, Yusong Wang +3
Recent advances in Large Language Models (LLMs) have opened new avenues for generating training-free text embeddings. However, the causal attention in decoder-only LLMs prevents ea…
Can LLMs Be Constrained to the Past? Improving Knowledge Cutoff through Recall-Based Prompting
Michiro Asai, Ailiang Lin, Yu Kishimoto +4
Prompted knowledge cutoff instructs a large language model (LLM) to act as if information beyond a specified cutoff date were unavailable. However, prior work mainly relies on dire…
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders
Ailiang Lin, Zhuoyun Li, Keyu Mao +2
Large language models (LLMs) have been widely explored for embedding generation. While recent studies show that in-context learning (ICL) effectively enhances the representational…
Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
Ailiang Lin, Zhuoyun Li, Yusong Wang +2
Decoder-only large language models (LLMs) have been increasingly adopted to build embedding models for diverse tasks. To overcome the inherent limitations of causal attention in re…