75 citations · 253 across the 38 of their papers we have counts for
5 papers · 1 filter
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
Shitao Xiao, Zheng Liu, Peitian Zhang +1
The pre-trained language models are continually fine-tuned to better support downstream applications. However, this operation may result in significant performance degeneration on…
Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval
Zheng Liu, Chaofan Li, Shitao Xiao +2
Dense retrieval calls for discriminative embeddings to represent the semantic relationship between query and document. It may benefit from the using of large language models (LLMs)…
A Multi-Task Embedder For Retrieval Augmented LLMs
Peitian Zhang, Shitao Xiao, Zheng Liu +2
LLMs confront inherent limitations in terms of its knowledge, memory, and action. The retrieval augmentation stands as a vital mechanism to address these limitations, which brings…
C-Pack: Packed Resources For General Chinese Embeddings
Shitao Xiao, Zheng Liu, Peitian Zhang +3
We introduce C-Pack, a package of resources that significantly advance the field of general Chinese embeddings. C-Pack includes three critical resources. 1) C-MTEB is a comprehensi…
RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
Shitao Xiao, Zheng Liu, Yingxia Shao +1
To better support information retrieval tasks such as web search and open-domain question answering, growing effort is made to develop retrieval-oriented language models, e.g., Ret…