32 citations · 276 across the 65 of their papers we have counts for
68 papers
VicEdit: Learning to Edit Videos from Visual In-Context Examples
Yuji Wang, Teng Hu, Yuheng Chen +6
Despite progress in instruction-based video editing, unimodal textual instructions inherently struggle to convey fine-grained textures and complex dynamics. To bridge this perceptu…
Disco-RAG: Discourse-Aware Retrieval-Augmented Generation
Dongqi Liu, Hang Ding, Qiming Feng +6
Retrieval-Augmented Generation (RAG) has emerged as an important means of enhancing the performance of large language models (LLMs) in knowledge-intensive tasks. However, most exis…
RoleRMBench & RoleRM: Towards Reward Modeling for Profile-Based Role Play in Dialogue Systems
Hang Ding, Qiming Feng, Dongqi Liu +9
Reward modeling has become a cornerstone of aligning large language models (LLMs) with human preferences. Yet, when extended to subjective and open-ended domains such as role play,…
IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment
Yinan Chen, Jiangning Zhang, Teng Hu +7
Instruction-guided video editing has emerged as a rapidly advancing research direction, offering new opportunities for intuitive content transformation while also posing significan…
Collaborative Face Experts Fusion in Video Generation: Boosting Identity Consistency Across Large Face Poses
Yuji Wang, Moran Li, Xiaobin Hu +7
Current video generation models struggle with identity preservation under large face poses, primarily facing two challenges: the difficulty in exploring an effective mechanism to i…
Semantic Frame Interpolation
Yijia Hong, Jiangning Zhang, Ran Yi +4
Generating intermediate video content of varying lengths based on given first and last frames, along with text prompt information, offers significant research and application poten…
HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding
Yuxuan Cai, Jiangning Zhang, Zhenye Gan +9
Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos. However, their capacity to comprehen…
Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations
Yuji Wang, Moran Li, Xiaobin Hu +7
Identity-preserving text-to-video (IPT2V) generation, which aims to create high-fidelity videos with consistent human identity, has become crucial for downstream applications. Howe…
Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning
Qingdong He, Xueqin Chen, Chaoyi Wang +7
Instruction-based image editing (IIE) has advanced rapidly with the success of diffusion models. However, existing efforts primarily focus on simple and explicit instructions to ex…
UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions
Zhucun Xue, Jiangning Zhang, Teng Hu +8
The quality of the video dataset (image quality, resolution, and fine-grained caption) greatly influences the performance of the video generation model. The growing demand for vide…
PiT: Progressive Diffusion Transformer
Jiafu Wu, Yabiao Wang, Jian Li +4
Diffusion Transformers (DiTs) achieve remarkable performance within image generation via the transformer architecture. Conventionally, DiTs are constructed by stacking serial isotr…
MARRS: Masked Autoregressive Unit-based Reaction Synthesis
Yabiao Wang, Shuo Wang, Jiangning Zhang +3
This work aims at a challenging task: human action-reaction synthesis, i.e., generating human reactions conditioned on the action sequence of another person. Currently, autoregress…
EMOv2: Pushing 5M Vision Model Frontier
Jiangning Zhang, Teng Hu, Haoyang He +6
This work focuses on developing parameter-efficient and lightweight models for dense predictions while trading off parameters, FLOPs, and performance. Our goal is to set up the new…
DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation
Qingdong He, Jinlong Peng, Pengcheng Xu +8
To enhance the controllability of text-to-image diffusion models, current ControlNet-like models have explored various control signals to dictate image attributes. However, existin…
MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
Haoyang He, Jiangning Zhang, Yuxuan Cai +7
Previous research on lightweight models has primarily focused on CNNs and Transformer-based designs. CNNs, with their local receptive fields, struggle to capture long-range depende…
Textual Decomposition Then Sub-motion-space Scattering for Open-Vocabulary Motion Generation
Ke Fan, Jiangning Zhang, Ran Yi +6
Text-to-motion generation is a crucial task in computer vision, which generates the target 3D motion by the given text. The existing annotated datasets are limited in scale, result…
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models
Yuxuan Cai, Jiangning Zhang, Haoyang He +7
The success of Large Language Models (LLMs) has inspired the development of Multimodal Large Language Models (MLLMs) for unified understanding of vision and language. However, the…
MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion
Chencan Fu, Yabiao Wang, Jiangning Zhang +7
Co-speech gesture generation is crucial for producing synchronized and realistic human gestures that accompany speech, enhancing the animation of lifelike avatars in virtual enviro…
MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation
Xiaofeng Mao, Zhengkai Jiang, Qilin Wang +7
Recent advancements in the field of Diffusion Transformers have substantially improved the generation of high-quality 2D images, 3D videos, and 3D shapes. However, the effectivenes…
LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description
Yizhang Jin, Jian Li, Jiangning Zhang +7
Visual Spatial Description (VSD) aims to generate texts that describe the spatial relationships between objects within images. Traditional visual spatial relationship classificatio…
Learning Multi-view Anomaly Detection with Efficient Adaptive Selection
Haoyang He, Jiangning Zhang, Guanzhong Tian +2
This study explores the recently proposed and challenging multi-view Anomaly Detection (AD) task. Single-view tasks will encounter blind spots from other perspectives, resulting in…
M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoising
Chengjie Wang, Haokun Zhu, Jinlong Peng +5
Existing industrial anomaly detection methods primarily concentrate on unsupervised learning with pristine RGB images. Yet, both RGB and 3D data are crucial for anomaly detection,…
ID-Sculpt: ID-aware 3D Head Generation from Single In-the-wild Portrait Image
Jinkun Hao, Junshu Tang, Jiangning Zhang +7
While recent works have achieved great success on image-to-3D object generation, high quality and fidelity 3D head generation from a single image remains a great challenge. Previou…
A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection
Jiangning Zhang, Haoyang He, Zhenye Gan +7
Visual anomaly detection aims to identify anomalous regions in images through unsupervised learning paradigms, with increasing application demand and value in fields such as indust…