6 papers
What Drives Interactive Improvement from Feedback?
Bartłomiej Cupiał, Jan Łojek, Mikołaj Garstecki +3
We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy c…
On the Role of Computation in Reinforcement Learning
Raj Ghugare, Michał Bortkiewicz, Alicja Ziarko +1
How does the amount of compute available to a reinforcement learning (RL) policy affect its learning? Can policies using a fixed amount of parameters, still benefit from additional…
Contrastive Representations for Temporal Reasoning
Alicja Ziarko, Michal Bortkiewicz, Michal Zawalski +2
In classical AI, perception relies on learning state-based representations, while planning, which can be thought of as temporal reasoning over action sequences, is typically achiev…
Beyond Recognition: Evaluating Visual Perspective Taking in Vision Language Models
Gracjan Góral, Alicja Ziarko, Piotr Miłoś +3
We investigate the ability of Vision Language Models (VLMs) to perform visual perspective taking using a new set of visual tasks inspired by established human tests. Our approach l…
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models
Gracjan Góral, Alicja Ziarko, Michal Nauman +1
Visual perspective-taking (VPT), the ability to understand the viewpoint of another person, enables individuals to anticipate the actions of other people. For instance, a driver ca…
Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
Alicja Ziarko, Albert Q. Jiang, Bartosz Piotrowski +3
Text embeddings are essential for many tasks, such as document retrieval, clustering, and semantic similarity assessment. In this paper, we study how to contrastively train text em…