9 papers
Can Vision Language Models Learn Intuitive Physics from Interaction?
Luca M. Schulze Buschoff, Konstantinos Voudouris, Can Demircan +1
Pre-trained vision language models do not have good intuitions about the physical world. Recent work has shown that supervised fine-tuning can improve model performance on simple p…
In-Context Function Learning in Large Language Models
Elif Akata, Konstantinos Voudouris, Vincent Fortuin +1
Large language models (LLMs) can learn from a few demonstrations provided at inference time. We study this in-context learning phenomenon through the lens of Gaussian Processes (GP…
Concept-Guided Interpretability via Neural Chunking
Shuchen Wu, Stephan Alaniz, Shyamgopal Karthik +3
Neural networks are often described as black boxes, reflecting the significant challenge of understanding their internal workings and interactions. We propose a different perspecti…
Building, Reusing, and Generalizing Abstract Representations from Concrete Sequences
Shuchen Wu, Mirko Thalmann, Peter Dayan +2
Humans excel at learning abstract patterns across different sequences, filtering out irrelevant details, and transferring these generalized concepts to new sequences. In contrast,…
Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models
Luca M. Schulze Buschoff, Konstantinos Voudouris, Elif Akata +3
Pre-trained vision language models still fall short of human visual cognition. In an effort to improve visual cognition and align models with human behavior, we introduce visual st…
Centaur: a foundation model of human cognition
Marcel Binz, Elif Akata, Matthias Bethge +37
Establishing a unified theory of cognition has been a major goal of psychology. While there have been previous attempts to instantiate such theories by building computational model…