48 citations · 57 across the 14 of their papers we have counts for
15 papers
TRACER: A Multimodal Harness for Patient Care in Hospitals
Zane Durante, Paul Tang, Keshigeyan Chandrasegaran +11
Continuous video and clinical records provide complementary information for understanding how patient behavior evolves over a hospital stay. A central challenge is how to represent…
VideoWeave: A Data-Centric Approach for Efficient Video Understanding
Zane Durante, Silky Singh, Arpandeep Khatua +6
Training video-language models is often prohibitively expensive due to the high cost of processing long frame sequences and the limited availability of annotated long videos. We pr…
Temporal Grounding as a Learning Signal for Referring Video Object Segmentation
Seunghun Lee, Jiwan Seo, Jeonghoon Kim +9
Referring Video Object Segmentation (RVOS) aims to segment and track objects in videos based on natural language expressions, requiring precise alignment between visual content and…
Latest Object Memory Management for Temporally Consistent Video Instance Segmentation
Seunghun Lee, Jiwan Seo, Minwoo Choi +6
In this paper, we present Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation that significantly improves long-term instance tracking. At t…
T*: Re-thinking Temporal Search for Long-Form Video Understanding
Jinhui Ye, Zihan Wang, Haosen Sun +9
Efficiently understanding long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding…
Towards Fine-Grained Video Question Answering
Wei Dai, Alan Luo, Zane Durante +5
In the rapidly evolving domain of video understanding, Video Question Answering (VideoQA) remains a focal point. However, existing datasets exhibit gaps in temporal and spatial gra…
HourVideo: 1-Hour Video-Language Understanding
Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic +7
We present HourVideo, a benchmark dataset for hour-long video-language understanding. Our dataset consists of a novel task suite comprising summarization, perception (recall, track…
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
Rylan Schaeffer, Dan Valentine, Luke Bailey +12
The integration of new modalities into frontier AI systems offers exciting capabilities, but also increases the possibility such systems can be adversarially manipulated in undesir…
Few-Shot Classification of Interactive Activities of Daily Living (InteractADL)
Zane Durante, Robathan Harries, Edward Vendrow +5
Understanding Activities of Daily Living (ADLs) is a crucial step for different applications including assistive robots, smart homes, and healthcare. However, to date, few benchmar…
An Interactive Agent Foundation Model
Zane Durante, Bidipta Sarkar, Ran Gong +17
The development of artificial intelligence systems is transitioning from creating static, task-specific models to dynamic, agent-based systems capable of performing well in a wide…
Agent AI: Surveying the Horizons of Multimodal Interaction
Zane Durante, Qiuyuan Huang, Naoki Wake +11
Multi-modal AI systems will likely become a ubiquitous presence in our everyday lives. A promising approach to making these systems more interactive is to embody them as agents wit…
MindAgent: Emergent Gaming Interaction
Ran Gong, Qiuyuan Huang, Xiaojian Ma +8
Large Language Models (LLMs) have the capacity of performing complex scheduling in a multi-agent system and can coordinate these agents into completing sophisticated tasks that req…
Differentially Private Video Activity Recognition
Zelun Luo, Yuliang Zou, Yijin Yang +6
In recent years, differential privacy has seen significant advancements in image classification; however, its application to video activity recognition remains under-explored. This…
Speech Representations and Phoneme Classification for Preserving the Endangered Language of Ladin
Zane Durante, Leena Mathur, Eric Ye +3
A vast majority of the world's 7,000 spoken languages are predicted to become extinct within this century, including the endangered language of Ladin from the Italian Alps. Linguis…
Deep CNN Frame Interpolation with Lessons Learned from Natural Language Processing
Kian Ghodoussi, Nihar Sheth, Zane Durante +1
A major area of growth within deep learning has been the study and implementation of convolutional neural networks. The general explanation within the deep learning community of th…