6 papers
Thinking Augmented Pre-training
Liang Wang, Nan Yang, Shaohan Huang +2
This paper introduces a simple and scalable approach to improve the data efficiency of large language model (LLM) training by augmenting existing text data with thinking trajectori…
Chain-of-Retrieval Augmented Generation
Liang Wang, Haonan Chen, Nan Yang +3
This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer. Conventional R…
Probabilistic Modeling of Disparity Uncertainty for Robust and Efficient Stereo Matching
Wenxiao Cai, Dongting Hu, Ruoyan Yin +4
Stereo matching plays a crucial role in various applications, where understanding uncertainty can enhance both safety and reliability. Despite this, the estimation and analysis of…
Generative Representational Instruction Tuning
Niklas Muennighoff, Hongjin Su, Liang Wang +5
All text-based language problems can be reduced to either generation or embedding. Current models only perform well at one or the other. We introduce generative representational in…
LongEmbed: Extending Embedding Models for Long Context Retrieval
Dawei Zhu, Liang Wang, Nan Yang +4
Embedding models play a pivot role in modern NLP applications such as IR and RAG. While the context limit of LLMs has been pushed beyond 1 million tokens, embedding models are stil…
Little Giants: Synthesizing High-Quality Embedding Data at Scale
Haonan Chen, Liang Wang, Nan Yang +4
Synthetic data generation has become an increasingly popular way of training models without the need for large, manually labeled datasets. For tasks like text embedding, synthetic…