402 citations · 611 across the 25 of their papers we have counts for
18 papers · 1 filter
On Locality and Length Generalization in Visual Reasoning
Pulkit Madan, Sanjay Haresh, Reza Ebrahimi +3
A striking feature of the human visual system is that it ingests visual information through a series of local foveated glimpses, rather than a single global computation. This makes…
Streaming Interventions: Can Video Large Language Models Correct Mistakes as They Occur?
Apratim Bhattacharyya, Shweta Mahajan, Sanjay Haresh +5
Learning everyday skills, like cooking a dish, relies increasingly on instructional media such as online videos. This opens the door to the use of video (and multimodal) large lang…
Generative Scenario Rollouts for End-to-End Autonomous Driving
Rajeev Yasarla, Deepti Hegde, Shizhong Han +10
Vision-Language-Action (VLA) models are emerging as highly effective planning models for end-to-end autonomous driving systems. However, current works mostly rely on imitation lear…
Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?
Apratim Bhattacharyya, Bicheng Xu, Sanjay Haresh +6
Multi-modal Large Language Models (LLM) have advanced conversational abilities but struggle with providing live, interactive step-by-step guidance, a key capability for future AI a…
RoCA: Robust Cross-Domain End-to-End Autonomous Driving
Rajeev Yasarla, Shizhong Han, Hsin-Pai Cheng +7
End-to-end (E2E) autonomous driving has recently emerged as a new paradigm, offering significant potential. However, few studies have looked into the practical challenge of deploym…
Can Vision-Language Models Answer Face to Face Questions in the Real-World?
Reza Pourreza, Rishit Dagli, Apratim Bhattacharyya +3
AI models have made significant strides in recent years in their ability to describe and answer questions about real-world images. They have also made progress in the ability to co…
Distilling Multi-modal Large Language Models for Autonomous Driving
Deepti Hegde, Rajeev Yasarla, Hong Cai +7
Autonomous driving demands safe motion planning, especially in critical "long-tail" scenarios. Recent end-to-end autonomous driving systems leverage large language models (LLMs) as…
What to Say and When to Say it: Live Fitness Coaching as a Testbed for Situated Interaction
Sunny Panchal, Apratim Bhattacharyya, Guillaume Berger +10
Vision-language models have shown impressive progress in recent years. However, existing models are largely limited to turn-based interactions, where each turn must be stepped (i.e…
Painter: Teaching Auto-regressive Language Models to Draw Sketches
Reza Pourreza, Apratim Bhattacharyya, Sunny Panchal +3
Large language models (LLMs) have made tremendous progress in natural language understanding and they have also been successfully adopted in other domains such as computer vision,…
Look, Remember and Reason: Grounded reasoning in videos with language models
Apratim Bhattacharyya, Sunny Panchal, Mingu Lee +3
Multi-modal language models (LM) have recently shown promising performance in high-level reasoning tasks on videos. However, existing methods still fall short in tasks like causal…
Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centers
Apratim Bhattacharyya, Daniel Olmeda Reino, Mario Fritz +1
Accurate prediction of pedestrian and bicyclist paths is integral to the development of reliable autonomous vehicles in dense urban environments. The interactions between vehicle a…
Haar Wavelet based Block Autoregressive Flows for Trajectories
Apratim Bhattacharyya, Christoph-Nikolas Straehle, Mario Fritz +1
Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents. While previous works have leveraged conditional generative models like GA…
Conditional Flow Variational Autoencoders for Structured Sequence Prediction
Apratim Bhattacharyya, Michael Hanselmann, Mario Fritz +2
Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world. Prior work for s…
Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods
Apratim Bhattacharyya, Mario Fritz, Bernt Schiele
For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence. In real-world scenarios, future states become in…
Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization
Apratim Bhattacharyya, Mario Fritz, Bernt Schiele
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence. This problem can be formalized a…
Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
Apratim Bhattacharyya, Mario Fritz, Bernt Schiele
Progress towards advanced systems for assisted and autonomous driving is leveraging recent advances in recognition and segmentation methods. Yet, we are still facing challenges in…
Long-Term Image Boundary Prediction
Apratim Bhattacharyya, Mateusz Malinowski, Bernt Schiele +1
Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone o…
Spatio-Temporal Image Boundary Extrapolation
Apratim Bhattacharyya, Mateusz Malinowski, Mario Fritz
Boundary prediction in images as well as video has been a very active topic of research and organizing visual information into boundaries and segments is believed to be a corner st…