211 citations · 1.2k across the 80 of their papers we have counts for
83 papers
FreeCHROMA: Generative Chroma Keying from Unaligned Background References
Jiayi Wu, Tianfu Wang, Rui Zhao +6
In film production, chroma keying composites an approved foreground performance into a new environment, but conventional workflows require coordinated background plates and dedicat…
Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation
Yuan Gan, Zongxin Yang, Xihang Yue +2
Audio-driven talking-head synthesis is a popular research topic for virtual human-related applications. However, the inflexibility and inefficiency of existing methods, which neces…
CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentation
Kexin Li, Zongxin Yang, Lei Chen +2
Audio-visual video segmentation~(AVVS) aims to generate pixel-level maps of sound-producing objects within image frames and ensure the maps faithfully adhere to the given audio, su…
DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion
Yunhong Lou, Linchao Zhu, Yaxiong Wang +2
We present DiverseMotion, a new approach for synthesizing high-quality human motions conditioned on textual descriptions while preserving motion diversity.Despite the recent signif…
MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems
Yu Gao, Lutong Su, Hao Liang +3
Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-cam…
Editing 3D Scenes via Text Prompts without Retraining
Shuangkang Fang, Yufeng Wang, Yi Yang +4
Numerous diffusion models have recently been applied to image synthesis and editing. However, editing 3D scenes is still in its early stages. It poses various challenges, such as t…
Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models
Yuanzhi Liang, Linchao Zhu, Yi Yang
Recent advancements in natural language and Large Language Models (LLMs) have enabled AI agents to simulate human-like interactions within virtual worlds. However, these interactio…
Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition
Chao Liang, Zongxin Yang, Linchao Zhu +1
In real-world scenarios, collected and annotated data often exhibit the characteristics of multiple classes and long-tailed distribution. Additionally, label noise is inevitable in…
Video Object Segmentation in Panoptic Wild Scenes
Yuanyou Xu, Zongxin Yang, Yi Yang
In this paper, we introduce semi-supervised video object segmentation (VOS) to panoptic wild scenes and present a large-scale benchmark as well as a baseline method for it. Previou…
Shuffled Autoregression For Motion Interpolation
Shuo Huang, Jia Jia, Zongxin Yang +4
This work aims to provide a deep-learning solution for the motion interpolation task. Previous studies solve it with geometric weight functions. Some other works propose neural net…
Whitening-based Contrastive Learning of Sentence Embeddings
Wenjie Zhuo, Yifan Sun, Xiaohan Wang +2
This paper presents a whitening-based contrastive learning method for sentence embedding learning (WhitenedCSE), which combines contrastive learning with a novel shuffled group whi…
Action Sensitivity Learning for the Ego4D Episodic Memory Challenge 2023
Jiayi Shao, Xiaohan Wang, Ruijie Quan +1
This report presents ReLER submission to two tracks in the Ego4D Episodic Memory Benchmark in CVPR 2023, including Natural Language Queries and Moment Queries. This solution inheri…
Gloss-Free End-to-End Sign Language Translation
Kezhou Lin, Xiaohan Wang, Linchao Zhu +3
In this paper, we tackle the problem of sign language translation (SLT) without gloss annotations. Although intermediate representation like gloss has been proven effective, gloss…
Efficient Multimodal Fusion via Interactive Prompting
Yaowei Li, Ruijie Quan, Linchao Zhu +1
Large-scale pre-training has brought unimodal fields such as computer vision and natural language processing to a new era. Following this trend, the size of multi-modal learning mo…
Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models
Shuai Zhao, Xiaohan Wang, Linchao Zhu +1
One fascinating aspect of pre-trained vision-language models~(VLMs) learning under language supervision is their impressive zero-shot generalization capability. However, this abili…
Action Sensitivity Learning for Temporal Action Localization
Jiayi Shao, Xiaohan Wang, Ruijie Quan +3
Temporal action localization (TAL), which involves recognizing and locating action instances, is a challenging task in video understanding. Most existing approaches directly predic…
CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model
Shuai Zhao, Ruijie Quan, Linchao Zhu +1
Pre-trained vision-language models~(VLMs) are the de-facto foundation models for various downstream tasks. However, scene text recognition methods still prefer backbones pre-traine…
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
Viorica Pătrăucean, Lucas Smaira, Ankush Gupta +21
We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e.g. Flamingo, SeViLA, or GP…
Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting
Jinliang Deng, Xiusi Chen, Renhe Jiang +4
Multivariate time-series (MTS) forecasting is a paramount and fundamental problem in many real-world applications. The core issue in MTS forecasting is how to effectively model com…
Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation
Xiaolong Shen, Zongxin Yang, Xiaohan Wang +3
Video-based 3D human pose and shape estimations are evaluated by intra-frame accuracy and inter-frame smoothness. Although these two metrics are responsible for different ranges of…
Lateral Contrast Enhancement in Tomographic Volumetric 3D-Printing via Binary Photoinhibition
Bin Wang, Weichao Sun Hossein Safari, S. Kaveh Hedayati +8
Tomographic volumetric 3D-printing (TVP) utilizes a nonlinear photoresponse of polymer precursor to cure all points in a three-dimensional (3D) object in parallel. A key challenge…
SN 2021zny: an early flux excess combined with late-time oxygen emission suggests a double white dwarf merger event
Georgios Dimitriadis, Kate Maguire, Viraj R. Karambelkar +28
We present a photometric and spectroscopic analysis of the ultra-luminous and slowly evolving 03fg-like Type Ia SN 2021zny. Our observational campaign starts from hours a…
Analogical Inference Enhanced Knowledge Graph Embedding
Zhen Yao, Wen Zhang, Mingyang Chen +3
Knowledge graph embedding (KGE), which maps entities and relations in a knowledge graph into continuous vector spaces, has achieved great success in predicting missing links in kno…
Knowledge-guided Causal Intervention for Weakly-supervised Object Localization
Feifei Shao, Yawei Luo, Fei Gao +2
Previous weakly-supervised object localization (WSOL) methods aim to expand activation map discriminative areas to cover the whole objects, yet neglect two inherent challenges when…