Publications (5)
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +306
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
SynLlama: Generating Synthesizable Molecules and Their Analogs with Large Language Models
Kunyang Sun, Dorian Bagni, Joseph M. Cavanagh +6
Generative machine learning models for exploring chemical space have shown immense promise, but many molecules they generate are too difficult to synthesize, making them impractica…
How Well Can Frontier Large Language Models Generate Structures? High Quality Prediction of Molecular Geometries with Help from Fine-Tuning
Joseph M. Cavanagh, Jonathan B. Arnold, Giovanni Battista Alteri +2
The paper explores fine‑tuning large language models to predict equilibrium geometries and conformers of small organic and drug‑like molecules, showing that Z‑matrix representation…
SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration
Joseph M. Cavanagh, Kunyang Sun, Andrew Gritsevskiy +4
We show that large language model (LLMs) can be transformed via supervised fine-tuning (SFT) of engineered prompts into SmileyLlama for exploring the chemical space of drug molecul…