17 papers
GestureFAR: Streaming Co-Speech Gesture Generation with Flow Autoregression
Pinxin Liu, Haiyang Liu, Jiahao Luo +3
Generating natural co-speech gestures from streaming speech is essential for embodied conversational agents, where motion must be produced while a user is still speaking. Recent st…
TDMM-LM: Bridging Facial Understanding and Animation via Language Models
Luchuan Song, Pinxin Liu, Haiyang Liu +7
Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage f…
Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
Yolo Y. Tang, Jing Bi, Pinxin Liu +24
Video understanding represents the most challenging frontier in computer vision, requiring models to reason about complex spatiotemporal relationships, long-term dependencies, and…
VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Results
Dasong Li, Sizhuo Ma, Hang Hua +40
This paper presents an overview of the VQualA 2025 Challenge on Engagement Prediction for Short Videos, held in conjunction with ICCV 2025. The challenge focuses on understanding a…
Can Sound Replace Vision in LLaVA With Token Substitution?
Ali Vosoughi, Jing Bi, Pinxin Liu +2
What happens when we push audio-visual alignment to its absolute limits? To systematically investigate this question, we needed datasets with granular alignment quality annotations…
: Generating Instructional Illustrations via Text-Conditioned Diffusion
Jing Bi, Pinxin Liu, Ali Vosoughi +3
The effective communication of procedural knowledge remains a significant challenge in natural language processing (NLP), as purely textual instructions often fail to convey comple…
MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness
Yolo Y. Tang, Pinxin Liu, Zhangyun Tan +11
Understanding perspective is fundamental to human visual perception, yet the extent to which multimodal large language models (MLLMs) internalize perspective geometry remains uncle…
Intentional Gesture: Deliver Your Intentions with Gestures for Speech
Pinxin Liu, Haiyang Liu, Luchuan Song +2
When humans speak, gestures help convey communicative intentions, such as adding emphasis or describing concepts. However, current co-speech gesture generation methods rely solely…
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting
Yunlong Tang, Jing Bi, Chao Huang +16
We present CAT-V (Caption AnyThing in Video), a training-free framework for fine-grained object-centric video captioning that enables detailed descriptions of user-selected objects…
Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)
Jing Bi, Susan Liang, Xiaofei Zhou +16
Reasoning is central to human intelligence, enabling structured problem-solving across diverse tasks. Recent advances in large language models (LLMs) have greatly enhanced their re…
GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling
Pinxin Liu, Luchuan Song, Junhua Huang +3
Generating full-body human gestures based on speech signals remains challenges on quality and speed. Existing approaches model different body regions such as body, legs and hands s…
Generative AI for Cel-Animation: A Survey
Yolo Y. Tang, Junjia Guo, Pinxin Liu +14
Traditional Celluloid (Cel) Animation production pipeline encompasses multiple essential steps, including storyboarding, layout design, keyframe animation, inbetweening, and colori…
TextToon: Real-Time Text Toonify Head Avatar from Single Video
Luchuan Song, Lele Chen, Celong Liu +2
We propose TextToon, a method to generate a drivable toonified avatar. Given a short monocular video sequence and a written instruction about the avatar style, our model can genera…
An Empirical Analysis on Large Language Models in Debate Evaluation
Xinyi Liu, Pinxin Liu, Hangfeng He
In this study, we investigate the capabilities and inherent biases of advanced large language models (LLMs) such as GPT-3.5 and GPT-4 in the context of debate evaluation. We discov…
Adaptive Super Resolution For One-Shot Talking-Head Generation
Luchuan Song, Pinxin Liu, Guojun Yin +1
The one-shot talking-head generation learns to synthesize a talking-head video with one source portrait image under the driving of same or different identity video. Usually these m…
GaussianStyle: Gaussian Head Avatar via StyleGAN
Pinxin Liu, Luchuan Song, Daoan Zhang +5
Existing methods like Neural Radiation Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made significant strides in facial attribute control such as facial animation and compone…
Tri-plane: Thinking Head Avatar via Feature Pyramid
Luchuan Song, Pinxin Liu, Lele Chen +2
Recent years have witnessed considerable achievements in facial avatar reconstruction with neural volume rendering. Despite notable advancements, the reconstruction of complex and…