24 citations · 57 across the 7 of their papers we have counts for
8 papers
How Well Can Frontier Large Language Models Generate Structures? High Quality Prediction of Molecular Geometries with Help from Fine-Tuning
Joseph M. Cavanagh, Jonathan B. Arnold, Giovanni Battista Alteri +2
The power of Large Language Models (LLMs) has led us to investigate how they might be fine-tuned for learning the "language of molecular geometry". The fine-tuning of the LLMs usin…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +378
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
SynLlama: Generating Synthesizable Molecules and Their Analogs with Large Language Models
Kunyang Sun, Dorian Bagni, Joseph M. Cavanagh +6
Generative machine learning models for exploring chemical space have shown immense promise, but many molecules they generate are too difficult to synthesize, making them impractica…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Traceable random numbers from a nonlocal quantum advantage
Gautam A. Kavuri, Jasper Palfree, Dileep V. Reddy +20
The unpredictability of random numbers is fundamental to both digital security and applications that fairly distribute resources. However, existing random number generators have li…
SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration
Joseph M. Cavanagh, Kunyang Sun, Andrew Gritsevskiy +4
We show that large language model (LLMs) can be transformed via supervised fine-tuning (SFT) of engineered prompts into SmileyLlama for exploring the chemical space of drug molecul…
REBUS: A Robust Evaluation Benchmark of Understanding Symbols
Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland +7
We propose a new benchmark evaluating the performance of multimodal large language models on rebus puzzles. The dataset covers 333 original examples of image-based wordplay, cluing…
Inverse Scaling: When Bigger Isn't Better
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler +24
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, w…