39 citations · 82 across the 19 of their papers we have counts for
7 papers · 2 filters
Bootstrap Your Own Context Length
Liang Wang, Nan Yang, Xingxing Zhang +2
We introduce a bootstrapping approach to train long-context language models by exploiting their short-context capabilities only. Our method utilizes a simple agent workflow to synt…
Little Giants: Synthesizing High-Quality Embedding Data at Scale
Haonan Chen, Liang Wang, Nan Yang +4
Synthetic data generation has become an increasingly popular way of training models without the need for large, manually labeled datasets. For tasks like text embedding, synthetic…
Pronunciation Assessment with Multi-modal Large Language Models
Kaiqi Fu, Linkai Peng, Nan Yang +1
Large language models (LLMs), renowned for their powerful conversational abilities, are widely recognized as exceptional tools in the field of education, particularly in the contex…
LongEmbed: Extending Embedding Models for Long Context Retrieval
Dawei Zhu, Liang Wang, Nan Yang +4
Embedding models play a pivot role in modern NLP applications such as IR and RAG. While the context limit of LLMs has been pushed beyond 1 million tokens, embedding models are stil…
Multilingual E5 Text Embeddings: A Technical Report
Liang Wang, Nan Yang, Xiaolong Huang +3
This technical report presents the training methodology and evaluation results of the open-source multilingual E5 text embedding models, released in mid-2023. Three embedding model…
Generative Representational Instruction Tuning
Niklas Muennighoff, Hongjin Su, Liang Wang +5
All text-based language problems can be reduced to either generation or embedding. Current models only perform well at one or the other. We introduce generative representational in…
Improving Text Embeddings with Large Language Models
Liang Wang, Nan Yang, Xiaolong Huang +3
In this paper, we introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. Unlike existing methods…