2 citations · 4 across the 12 of their papers we have counts for
13 papers
DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation
Yubo Huang, Sirui Zhao, Xinchen Yao +5
Audio-driven avatar generation requires realistic lip-sync, expressive motion, and real-time streaming. Recent work achieves the latter via self-forcing with Distribution Matching…
ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning
Shifeng Liu, Zhengye Zhang, Sirui Zhao +7
Recent advances in Multimodal Large Language Models (MLLMs) have created new opportunities for facial expression recognition (FER), moving it beyond pure label prediction toward re…
Benchmarking Large Language Models for Conversational Question Answering in Multi-instructional Documents
Shiwei Wu, Chen Zhang, Yan Gao +4
Instructional documents are rich sources of knowledge for completing various tasks, yet their unique challenges in conversational question answering (CQA) have not been thoroughly…
Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal Summarization
Yanghai Zhang, Ye Liu, Shiwei Wu +4
The rapid increase in multimedia data has spurred advancements in Multimodal Summarization with Multimodal Output (MSMO), which aims to produce a multimodal summary that integrates…
From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition
Shiwei Wu, Chao Zhang, Joya Chen +4
People's social relationships are often manifested through their surroundings, with certain objects or interactions acting as symbols for specific relationships, e.g., wedding ring…
NoteLLM-2: Multimodal Large Representation Models for Recommendation
Chao Zhang, Haoxin Zhang, Shiwei Wu +6
Large Language Models (LLMs) have demonstrated exceptional proficiency in text understanding and embedding tasks. However, their potential in multimodal representation, particularl…
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Chao Zhang, Shiwei Wu, Haoxin Zhang +5
People enjoy sharing "notes" including their experiences within online communities. Therefore, recommending notes aligned with user interests has become a crucial task. Existing on…
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
Yifei Cheng, Li Shen, Linli Xu +6
Gradient compression with error compensation has attracted significant attention with the target of reducing the heavy communication overhead in distributed learning. However, exis…
Multi-Grained Multimodal Interaction Network for Entity Linking
Pengfei Luo, Tong Xu, Shiwei Wu +3
Multimodal entity linking (MEL) task, which aims at resolving ambiguous mentions to a multimodal knowledge graph, has attracted wide attention in recent years. Though large efforts…
A Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inference
Chao Zhang, Shiwei Wu, Sirui Zhao +2
Affordance-centric Question-driven Task Completion (AQTC) for Egocentric Assistant introduces a groundbreaking scenario. In this scenario, through learning instructional videos, AI…
AU-aware graph convolutional network for Macro- and Micro-expression spotting
Shukang Yin, Shiwei Wu, Tong Xu +3
Automatic Micro-Expression (ME) spotting in long videos is a crucial step in ME analysis but also a challenging task due to the short duration and low intensity of MEs. When solvin…
Winning the CVPR'2022 AQTC Challenge: A Two-stage Function-centric Approach
Shiwei Wu, Weidong He, Tong Xu +2
Affordance-centric Question-driven Task Completion for Egocentric Assistant(AQTC) is a novel task which helps AI assistant learn from instructional videos and scripts and guide the…
Is Heuristic Sampling Necessary in Training Deep Object Detectors?
Joya Chen, Dong Liu, Tong Xu +3
To train accurate deep object detectors under the extreme foreground-background imbalance, heuristic sampling methods are always necessary, which either re-sample a subset of all t…