7 citations · 26 across the 34 of their papers we have counts for
9 papers · 1 filter
How to Merge Your Multimodal Models Over Time?
Sebastian Dziadzio, Vishaal Udandarao, Karsten Roth +4
Model merging combines multiple expert models - finetuned from a base foundation model on diverse tasks and domains - into a single, more capable model. However, most existing mode…
ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities
Adhiraj Ghosh, Sebastian Dziadzio, Ameya Prabhu +3
Traditional fixed test sets fall short in evaluating open-ended capabilities of foundation models. To address this, we propose ONEBench(OpeN-Ended Benchmarking), a new testing para…
Centaur: a foundation model of human cognition
Marcel Binz, Elif Akata, Matthias Bethge +37
Establishing a unified theory of cognition has been a major goal of psychology. While there have been previous attempts to instantiate such theories by building computational model…
A Practitioner's Guide to Continual Multimodal Pretraining
Karsten Roth, Vishaal Udandarao, Sebastian Dziadzio +7
Multimodal foundation models serve numerous applications at the intersection of vision and language. Still, despite being pretrained on extensive data, they become outdated over ti…
CiteME: Can Language Models Accurately Cite Scientific Claims?
Ori Press, Andreas Hochlehnert, Ameya Prabhu +3
Thousands of new scientific papers are published each month. Such information overload complicates researcher efforts to stay current with the state-of-the-art as well as to verify…
The Entropy Enigma: Success and Failure of Entropy Minimization
Ori Press, Ravid Shwartz-Ziv, Yann LeCun +1
Entropy minimization (EM) is frequently used to increase the accuracy of classification models when they're faced with new data at test time. EM is a self-supervised learning metho…
Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry
Shiven Sinha, Ameya Prabhu, Ponnurangam Kumaraguru +2
Proving geometric theorems constitutes a hallmark of visual reasoning combining both intuitive and logical skills. Therefore, automated theorem proving of Olympiad-level geometry p…
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh +5
Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for…
Efficient Lifelong Model Evaluation in an Era of Rapid Progress
Ameya Prabhu, Vishaal Udandarao, Philip Torr +3
Standardized benchmarks drive progress in machine learning. However, with repeated testing, the risk of overfitting grows as algorithms over-exploit benchmark idiosyncrasies. In ou…