17 citations · 44 across the 14 of their papers we have counts for
16 papers
Multimodal Generative Recommendation for Fusing Semantic and Collaborative Signals
Moritz Vandenhirtz, Kaveh Hassani, Shervin Ghasemlou +5
Sequential recommender systems rank relevant items by modeling a user's interaction history and computing the inner product between the resulting user representation and stored ite…
Synthetic Captions for Open-Vocabulary Zero-Shot Segmentation
Tim Lebailly, Vijay Veerabadran, Satwik Kottur +2
Generative vision-language models (VLMs) exhibit strong high-level image understanding but lack spatially dense alignment between vision and language modalities, as our findings in…
Enhancing Visual Planning with Auxiliary Tasks and Multi-token Prediction
Ce Zhang, Yale Song, Ruta Desai +4
Visual Planning for Assistance (VPA) aims to predict a sequence of user actions required to achieve a specified goal based on a video showing the user's progress. Although recent a…
EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos
Yuxuan Li, Vijay Veerabadran, Michael L. Iuzzolino +3
We introduce EgoToM, a new video question-answering benchmark that extends Theory-of-Mind (ToM) evaluation to egocentric domains. Using a causal ToM model, we generate multi-choice…
Less or More: Towards Glanceable Explanations for LLM Recommendations Using Ultra-Small Devices
Xinru Wang, Mengjie Yu, Hannah Nguyen +10
Large Language Models (LLMs) have shown remarkable potential in recommending everyday actions as personal AI assistants, while Explainable AI (XAI) techniques are being increasingl…
Multiscale Video Pretraining for Long-Term Activity Forecasting
Reuben Tan, Matthias De Lange, Michael Iuzzolino +4
Long-term activity forecasting is an especially challenging research problem because it requires understanding the temporal relationships between observed actions, as well as the v…
EgoAdapt: A multi-stream evaluation study of adaptation to real-world egocentric user video
Matthias De Lange, Hamid Eghbalzadeh, Reuben Tan +3
In egocentric action recognition a single population model is typically trained and subsequently embodied on a head-mounted device, such as an augmented reality headset. While this…
Pretrained Language Models as Visual Planners for Human Assistance
Dhruvesh Patel, Hamid Eghbalzadeh, Nitin Kamra +3
In our pursuit of advancing multi-modal AI assistants capable of guiding users to achieve complex multi-step goals, we propose the task of "Visual Planning for Assistance (VPA)". G…
Action Dynamics Task Graphs for Learning Plannable Representations of Procedural Tasks
Weichao Mao, Ruta Desai, Michael Louis Iuzzolino +1
Given video demonstrations and paired narrations of an at-home procedural task such as changing a tire, we present an approach to extract the underlying task structure -- relevant…
Online Unsupervised Learning of Visual Representations and Categories
Mengye Ren, Tyler R. Scott, Michael L. Iuzzolino +2
Real world learning scenarios involve a nonstationary distribution of classes with sequential dependencies among the samples, in contrast to the standard machine learning formulati…
Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss
Michael L. Iuzzolino, Michael C. Mozer, Samy Bengio
Although deep feedforward neural networks share some characteristics with the primate visual system, a key distinction is their dynamics. Deep nets typically operate in serial stag…
Wandering Within a World: Online Contextualized Few-Shot Learning
Mengye Ren, Michael L. Iuzzolino, Michael C. Mozer +1
We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting. In this s…
In Automation We Trust: Investigating the Role of Uncertainty in Active Learning Systems
Michael L. Iuzzolino, Tetsumichi Umada, Nisar R. Ahmed +1
We investigate how different active learning (AL) query policies coupled with classification uncertainty visualizations affect analyst trust in automated classification systems. A…
MMTM: Multimodal Transfer Module for CNN Fusion
Hamid Reza Vaezi Joze, Amirreza Shaban, Michael L. Iuzzolino +1
In late fusion, each modality is processed in a separate unimodal Convolutional Neural Network (CNN) stream and the scores of each modality are fused at the end. Due to its simplic…
Convolutional Bipartite Attractor Networks
Michael Iuzzolino, Yoram Singer, Michael C. Mozer
In human perception and cognition, a fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambi…
Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails
Michael L. Iuzzolino, Michael E. Walker, Daniel Szafir
Robots hold promise in many scenarios involving outdoor use, such as search-and-rescue, wildlife management, and collecting data to improve environment, climate, and weather foreca…