4 papers
Keep The Essentials: Efficient Reference Conditioned Generation via Token Dropping
Rishubh Parihar, Ayush Raina, R. Venkatesh Babu +1
Reference-based diffusion models enable highly controllable image generation by leveraging elements from input images to guide prompt-driven synthesis. However, these models are co…
The Dual Mechanisms of Spatial Variable Binding in Vision-Language Models
Kelly Cui, Nikhil Prakash, Shoval Messica +4
Many multimodal tasks, such as image captioning and visual question answering, require vision-language models (VLMs) to bind objects with their properties and spatial relations. Ye…
VirtualEnv: A Platform for Embodied AI Research
Kabir Swain, Sijie Han, Ayush Raina +4
As large language models (LLMs) continue to improve in reasoning and decision-making, there is a growing need for realistic and interactive environments where their abilities can b…
Behavioural Cloning in VizDoom
Ryan Spick, Timothy Bradley, Ayush Raina +2
This paper describes methods for training autonomous agents to play the game "Doom 2" through Imitation Learning (IL) using only pixel data as input. We also explore how Reinforcem…