5 papers
Uncovering Scaling Laws for Large Language Models via Inverse Problems
Arun Verma, Zhaoxuan Wu, Zijian Zhou +15
Large Language Models (LLMs) are large-scale pretrained models that have achieved remarkable success across diverse domains. These successes have been driven by unprecedented compl…
ReasonIR: Training Retrievers for Reasoning Tasks
Rulin Shao, Rui Qiao, Varsha Kishore +8
We present ReasonIR-8B, the first retriever specifically trained for general reasoning tasks. Existing retrievers have shown limited gains on reasoning tasks, in part because exist…
Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions
Rui Qiao, Zhaoxuan Wu, Jingtan Wang +2
Machine learning models often have uneven performance among subpopulations (a.k.a., groups) in the data distributions. This poses a significant challenge for the models to generali…
Data-Centric AI in the Age of Large Language Models
Xinyi Xu, Zhaoxuan Wu, Rui Qiao +16
This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs). We start by making the key observation that data is instrumental in…
Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions
Jingtan Wang, Xiaoqiang Lin, Rui Qiao +2
The increasing complexity of foundational models underscores the necessity for explainability, particularly for fine-tuning, the most widely used training method for adapting model…