activity
20222026
most citedHolistic Evaluation of Language Models

128 citations · 129 across the 8 of their papers we have counts for

collaborators

8 papers

cs.LG2026

Contrastive Concept-Tree Search for LLM-Assisted Algorithm Discovery

Timothee Leleu, Sudeera Gunathilaka, Federico Ghimenti +1

Large language Model (LLM)-assisted algorithm discovery is an iterative, black-box optimization process over programs to approximatively solve a target task, where an LLM proposes…

cond-mat.dis-nn2026

Reshaping Global Loop Structure to Accelerate Local Optimization by Smoothing Rugged Landscapes

Timothee Leleu, Sam Reifenstein, Atsushi Yamamura +1

Probabilistic graphical models with frustration exhibit rugged energy landscapes that trap iterative optimization dynamics. These landscapes are shaped not only by local interactio…

q-bio.NC2025

Short-term plasticity recalls forgotten memories through a trampoline mechanism

Martina Del Gaudio, Federico Ghimenti, Surya Ganguli

We analyze continuous Hopfield associative memories augmented by additional, rapid short-term associative synaptic plasticity. Through the cavity method, we determine the boundary…

cond-mat.dis-nn2025★ 1 cited

The geometry and dynamics of annealed optimization in the coherent Ising machine with hidden and planted solutions

Federico Ghimenti, Adithya Sriram, Atsushi Yamamura +2

The coherent Ising machine (CIM) is a nonconventional hardware architecture for finding approximate solutions to large-scale combinatorial optimization problems. It operates by ann…

cs.LG2025

CMT-Benchmark: A Benchmark for Condensed Matter Theory Built by Expert Researchers

Haining Pan, James V. Roggeveen, Erez Berg +16

Large language models (LLMs) have shown remarkable progress in coding and math problem-solving, but evaluation on advanced research-level problems in hard sciences remains scarce.…

cs.CR2024

Fooling LLM graders into giving better grades through neural activity guided adversarial prompting

Atsushi Yamamura, Surya Ganguli

The deployment of artificial intelligence (AI) in critical decision-making and evaluation processes raises concerns about inherent biases that malicious actors could exploit to dis…

cond-mat.dis-nn2023

Geometric landscape annealing as an optimization principle underlying the coherent Ising machine

Atsushi Yamamura, Hideo Mabuchi, Surya Ganguli

Given the fundamental importance of combinatorial optimization across many diverse application domains, there has been widespread interest in the development of unconventional phys…

cs.CL2022★ 128 cited

Holistic Evaluation of Language Models

Percy Liang, Rishi Bommasani, Tony Lee +47

Language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks are not well understood. We present Hol…