55 citations · 190 across the 22 of their papers we have counts for
28 papers
Interact with me: Joint Egocentric Forecasting of Intent to Interact, Attitude and Social Actions
Tongfei Bian, Yiming Ma, Mathieu Chollet +2
For efficient human-agent interaction, an agent should proactively recognize their target user and prepare for upcoming interactions. We formulate this challenging problem as the n…
Active Listener: Continuous Generation of Listener's Head Motion Response in Dyadic Interactions
Bishal Ghosh, Emma Li, Tanaya Guha
A key component of dyadic spoken interactions is the contextually relevant non-verbal gestures, such as head movements that reflect a listener's response to the interlocutor's spee…
WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting
Olly Styles, Sam Miller, Patricio Cerda-Mardini +3
We introduce WorkBench: a benchmark dataset for evaluating agents' ability to execute tasks in a workplace setting. WorkBench contains a sandbox environment with five databases, 26…
CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification
Yiming Ma, Victor Sanchez, Tanaya Guha
We propose CLIP-EBC, the first fully CLIP-based model for accurate crowd density estimation. While the CLIP model has demonstrated remarkable success in addressing recognition task…
Explainable Depression Detection via Head Motion Patterns
Monika Gahalawat, Raul Fernandez Rojas, Tanaya Guha +2
While depression has been studied via multimodal non-verbal behavioural cues, head motion behaviour has not received much attention as a biomarker. This study demonstrates the util…
Robust Multiview Multimodal Driver Monitoring System Using Masked Multi-Head Self-Attention
Yiming Ma, Victor Sanchez, Soodeh Nikan +3
Driver Monitoring Systems (DMSs) are crucial for safe hand-over actions in Level-2+ self-driving vehicles. State-of-the-art DMSs leverage multiple sensors mounted at different loca…
Heterogeneous Graph Learning for Acoustic Event Classification
Amir Shirian, Mona Ahmadian, Krishna Somandepalli +1
Heterogeneous graphs provide a compact, efficient, and scalable way to model data involving multiple disparate modalities. This makes modeling audiovisual data using heterogeneous…
Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics
Surbhi Madan, Monika Gahalawat, Tanaya Guha +2
We explore the efficacy of multimodal behavioral cues for explainable prediction of personality and interview-specific traits. We utilize elementary head-motion units named kinemes…
Real-Time Driver Monitoring Systems through Modality and View Analysis
Yiming Ma, Victor Sanchez, Soodeh Nikan +3
Driver distractions are known to be the dominant cause of road accidents. While monitoring systems can detect non-driving-related activities and facilitate reducing the risks, they…
Visually-aware Acoustic Event Detection using Heterogeneous Graphs
Amir Shirian, Krishna Somandepalli, Victor Sanchez +1
Perception of auditory events is inherently multimodal relying on both audio and visual cues. A large number of existing multimodal approaches process each modality using modality-…
Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
Kyle Min, Sourya Roy, Subarna Tripathi +2
Active speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over…
FusionCount: Efficient Crowd Counting via Multiscale Feature Fusion
Yiming Ma, Victor Sanchez, Tanaya Guha
State-of-the-art crowd counting models follow an encoder-decoder approach. Images are first processed by the encoder to extract features. Then, to account for perspective distortio…
Self-supervised Graphs for Audio Representation Learning with Limited Labeled Data
Amir Shirian, Krishna Somandepalli, Tanaya Guha
Large scale databases with high-quality manual annotations are scarce in audio domain. We thus explore a self-supervised graph approach to learning audio representations from highl…
Head Matters: Explainable Human-centered Trait Prediction from Head Motion Dynamics
Surbhi Madan, Monika Gahalawat, Tanaya Guha +1
We demonstrate the utility of elementary head-motion units termed kinemes for behavioral analytics to predict personality and interview traits. Transforming head-motion patterns in…
Learning Spatial-Temporal Graphs for Active Speaker Detection
Sourya Roy, Kyle Min, Subarna Tripathi +2
We address the problem of active speaker detection through a new framework, called SPELL, that learns long-range multimodal graphs to encode the inter-modal relationship between au…
Multi-Camera Trajectory Forecasting with Trajectory Tensors
Olly Styles, Tanaya Guha, Victor Sanchez
We introduce the problem of multi-camera trajectory forecasting (MCTF), which involves predicting the trajectory of a moving object across a network of cameras. While multi-camera…
In Defense of Scene Graphs for Image Captioning
Kien Nguyen, Subarna Tripathi, Bang Du +2
The mainstream image captioning models rely on Convolutional Neural Network (CNN) image features to generate captions via recurrent models. Recently, image scene graphs have been u…
Compact Graph Architecture for Speech Emotion Recognition
A. Shirian, T. Guha
We propose a deep graph approach to address the task of speech emotion recognition. A compact, efficient and scalable way to represent data is in the form of graphs. Following the…
Dynamic Character Graph via Online Face Clustering for Movie Analysis
Prakhar Kulshreshtha, Tanaya Guha
An effective approach to automated movie content analysis involves building a network (graph) of its characters. Existing work usually builds a static character graph to summarize…
Ensemble Network for Ranking Images Based on Visual Appeal
Sachin Singh, Victor Sanchez, Tanaya Guha
We propose a computational framework for ranking images (group photos in particular) taken at the same event within a short time span. The ranking is expected to correspond with hu…
Multi-Camera Trajectory Forecasting: Pedestrian Trajectory Prediction in a Network of Cameras
Olly Styles, Tanaya Guha, Victor Sanchez +1
We introduce the task of multi-camera trajectory forecasting (MCTF), where the future trajectory of an object is predicted in a network of cameras. Prior works consider forecasting…
Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zeroshot Classification and Retrieval of Videos
Kranti Kumar Parida, Neeraj Matiyali, Tanaya Guha +1
We present an audio-visual multimodal approach for the task of zeroshot learning (ZSL) for classification and retrieval of videos. ZSL has been studied extensively in the recent pa…
Multiple Object Forecasting: Predicting Future Object Locations in Diverse Environments
Olly Styles, Tanaya Guha, Victor Sanchez
This paper introduces the problem of multiple object forecasting (MOF), in which the goal is to predict future bounding boxes of tracked objects. In contrast to existing works on o…
Learning Affective Correspondence between Music and Image
Gaurav Verma, Eeshan Gunesh Dhekane, Tanaya Guha
We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (havin…