4 papers
Language Model Uncertainty Quantification with Attention Chain
Yinghao Li, Rushi Qiang, Lama Moukheiber +1
Accurately quantifying a large language model's (LLM) predictive uncertainty is crucial for judging the reliability of its answers. While most existing research focuses on short, d…
MSQA: Benchmarking LLMs on Graduate-Level Materials Science Reasoning and Knowledge
Jerry Junyang Cheung, Shiyao Shen, Yuchen Zhuang +3
Despite recent advances in large language models (LLMs) for materials science, there is a lack of benchmarks for evaluating their domain-specific knowledge and complex reasoning ab…
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
Rushi Qiang, Yuchen Zhuang, Yinghao Li +8
We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents in iterative machine…
Ensembles of Low-Rank Expert Adapters
Yinghao Li, Vianne Gao, Chao Zhang +1
The training and fine-tuning of large language models (LLMs) often involve diverse textual data from multiple sources, which poses challenges due to conflicting gradient directions…