5 citations · 13 across the 11 of their papers we have counts for
12 papers
ArtifactArena: Evaluating Models by What They Build in the Physical World
Kushagra Tiwary, Kushagra Tiwary*, David Mayo* +7
To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an ope…
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
Measuring Progress Toward AGI: A Cognitive Framework
Ryan Burnell, Yumeya Yamamori, Orhan Firat +10
Despite widespread discussion of AGI, there is no clear framework for measuring progress toward it. This ambiguity fuels subjective claims, makes it difficult to track progress, an…
Uneven Evolution of Cognition Across Generations of Generative AI Models
Isaac Galatzer-Levy, Daniel McDuff, Xin Liu +1
The pursuit of artificial general intelligence necessitates robust methods for evaluating the cognitive capabilities of models beyond narrow task performance. Here, we introduce a…
Towards a General Intelligence and Interface for Wearable Health Data
Girish Narayanswamy, Maxwell A. Xu, A. Ali Heydari +37
While ubiquitous wearable sensors capture a wealth of behavioral and physiological information, effectively transforming these signals into personalized health insights is challeng…
The Anatomy of a Personal Health Agent
A. Ali Heydari, Ken Gu, Vidya Srinivas +35
Health is a fundamental pillar of human wellness, and the rapid advancements in large language models (LLMs) have driven the development of a new generation of health agents. Howev…
Passive Measurement of Autonomic Arousal in Real-World Settings
Samy Abdel-Ghaffar, Isaac Galatzer-Levy, Conor Heneghan +5
The autonomic nervous system (ANS) is activated during stress, which can have negative effects on cardiovascular health, sleep, the immune system, and mental health. While there ar…
Optimizing Large Language Models for Detecting Symptoms of Comorbid Depression or Anxiety in Chronic Diseases: Insights from Patient Messages
Jiyeong Kim, Stephen P. Ma, Michael L. Chen +8
Patients with diabetes are at increased risk of comorbid depression or anxiety, complicating their management. This study evaluated the performance of large language models (LLMs)…
Substance over Style: Evaluating Proactive Conversational Coaching Agents
Vidya Srinivas, Xuhai Xu, Xin Liu +5
While NLP research has made strides in conversational tasks, many approaches focus on single-turn responses with well-defined objectives or evaluation criteria. In contrast, coachi…
Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions
Taedong Yun, Eric Yang, Mustafa Safdari +13
We present an end-to-end framework for generating synthetic users for evaluating interactive agents designed to encourage positive behavior changes, such as in health and lifestyle…
Evidence of Cognitive Deficits andDevelopmental Advances in Generative AI: A Clock Drawing Test Analysis
Isaac R. Galatzer-Levy, Jed McGiffin, David Munday +6
Generative AI's rapid advancement sparks interest in its cognitive abilities, especially given its capacity for tasks like language understanding and code generation. This study ex…
The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
Isaac R. Galatzer-Levy, David Munday, Jed McGiffin +6
There is increasing interest in tracking the capabilities of general intelligence foundation models. This study benchmarks leading large language models and vision language models…