2 citations · 5 across the 17 of their papers we have counts for
11 papers · 1 filter
How Much Future Helps? A Controlled Study of Future-Privileged Supervision for Causal Egocentric Gaze Estimation
Jia Li, Wenjie Zhao, Fnu Atisri +5
Egocentric gaze estimation is commonly studied using models that process the full video with access to future frames, while real-world applications require strictly causal, online…
Audio-Visual Intelligence in Large Foundation Models
You Qin, Kai Liu, Shengqiong Wu +12
Audio-Visual Intelligence (AVI) has emerged as a central frontier in artificial intelligence, bridging auditory and visual modalities to enable machines that can perceive, generate…
Omni-MMSI: Toward Identity-attributed Social Interaction Understanding
Xinpeng Li, Bolin Lai, Hardy Chen +5
We introduce Omni-MMSI, a new task that requires comprehensive social interaction understanding from raw audio, vision, and speech input. The task involves perceiving identity-attr…
ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
Jia Li, Wenjie Zhao, Shijian Deng +6
Online egocentric gaze estimation predicts where a camera wearer is looking from first-person video using only past and current frames, a task essential for augmented reality and a…
Explainable AI-Generated Image Detection RewardBench
Michael Yang, Shijian Deng, William T. Doan +4
Conventional, classification-based AI-generated image detection methods cannot explain why an image is considered real or AI-generated in a way a human expert would, which reduces…
Toward Gaze Target Detection of Young Autistic Children
Shijian Deng, Erin E. Kosloski, Siva Sai Nagender Vasireddy +6
The automatic detection of gaze targets in autistic children through artificial intelligence can be impactful, especially for those who lack access to a sufficient number of profes…
Towards Online Multi-Modal Social Interaction Understanding
Xinpeng Li, Shijian Deng, Bolin Lai +3
In this paper, we introduce a new problem, Online-MMSI, where the model must perform multimodal social interaction understanding (MMSI) using only historical information. Given a r…
Continual Audio-Visual Sound Separation
Weiguo Pian, Yiyang Nan, Shijian Deng +3
In this paper, we introduce a novel continual audio-visual sound separation task, aiming to continuously separate sound sources for new classes while preserving performance on prev…
AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation
Kai Wang, Shijian Deng, Jing Shi +2
Recent Diffusion Transformers (DiTs) have shown impressive capabilities in generating high-quality single-modality content, including images, videos, and audio. However, it is stil…
Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation
Yiyang Su, Ali Vosoughi, Shijian Deng +2
The audio-visual sound separation field assumes visible sources in videos, but this excludes invisible sounds beyond the camera's view. Current methods struggle with such sounds la…
Unveiling Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA
Ali Vosoughi, Shijian Deng, Songyang Zhang +3
To increase the generalization capability of VQA systems, many recent studies have tried to de-bias spurious language or vision associations that shortcut the question or image to…