Publications (8)
Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations
Sangmin Lee, Bolin Lai, Fiona Ryan +2
Understanding social interactions involving both verbal and non-verbal cues is essential for effectively interpreting social situations. However, most prior works on multimodal soc…
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric…
Multipath Graph Convolutional Neural Networks
Rangan Das, Bikram Boote, Saumik Bhattacharya +1
Graph convolution networks have recently garnered a lot of attention for representation learning on non-Euclidean feature spaces. Recent research has focused on stacking multiple l…
Leveraging Object Priors for Point Tracking
Bikram Boote, Anh Thai, Wenqi Jia +4
Point tracking is a fundamental problem in computer vision with numerous applications in AR and robotics. A common failure mode in long-term point tracking occurs when the predicte…
Towards Social AI: A Survey on Understanding Social Interactions
Sangmin Lee, Minzhi Li, Bolin Lai +8
Social interactions form the foundation of human societies. Artificial intelligence has made significant progress in certain areas, but enabling machines to seamlessly understand s…
GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interactions
Junho Kim, Xu Cao, Houze Yang +6
Understanding social interactions requires reasoning over subtle non-verbal cues, yet current multimodal large language models (MLLMs) often fail to identify who interacts with who…
MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models
Anh Thai, Stefan Stojanov, Zixuan Huang +2
This paper introduces MEBench, a novel benchmark for evaluating mutual exclusivity (ME) bias, a cognitive phenomenon observed in children during word learning. Unlike traditional M…
Low-shot Object Learning with Mutual Exclusivity Bias
Anh Thai, Ahmad Humayun, Stefan Stojanov +3
This paper introduces Low-shot Object Learning with Mutual Exclusivity Bias (LSME), the first computational framing of mutual exclusivity bias, a phenomenon commonly observed in in…