57 citations · 186 across the 14 of their papers we have counts for
14 papers
Cross-modal Contrastive Distillation for Instructional Activity Anticipation
Zhengyuan Yang, Jingen Liu, Jing Huang +4
In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation. Unlike previous antic…
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption
Zhengyuan Yang, Yijuan Lu, Jianfeng Wang +6
In this paper, we propose Text-Aware Pre-training (TAP) for Text-VQA and Text-Caption tasks. These two tasks aim at reading and understanding scene text in images for question answ…
Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation
Zhengyuan Yang, Amanda Kay, Yuncheng Li +2
Inspired by the human ability to infer emotions from body language, we propose an automated framework for body language based emotion recognition starting from regular RGB videos.…
Dynamic Context-guided Capsule Network for Multimodal Machine Translation
Huan Lin, Fandong Meng, Jinsong Su +5
Multimodal machine translation (MMT), which mainly focuses on enhancing text-only translation with visual features, has attracted considerable attention from both computer vision a…
Improving One-stage Visual Grounding by Recursive Sub-query Construction
Zhengyuan Yang, Tianlang Chen, Liwei Wang +1
We improve one-stage visual grounding by addressing current limitations on grounding long and complex queries. Existing one-stage methods encode the entire language query as a sing…
A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation
Yongjing Yin, Fandong Meng, Jinsong Su +4
Multi-modal neural machine translation (NMT) aims to translate source sentences into a target language paired with images. However, dominant multi-modal NMT models do not fully exp…
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
Liwei Wang, Jing Huang, Yin Li +3
Weakly supervised phrase grounding aims at learning region-phrase correspondences using only image-sentence pairs. A major challenge thus lies in the missing links between image re…
Grounding-Tracking-Integration
Zhengyuan Yang, Tushar Kumar, Tianlang Chen +2
In this paper, we study Tracking by Language that localizes the target box sequence in a video based on a language query. We propose a framework called GTI that decomposes the prob…
A Fast and Accurate One-Stage Approach to Visual Grounding
Zhengyuan Yang, Boqing Gong, Liwei Wang +3
We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight. The performances of existing propose-and-rank two-stage methods a…
Weakly Supervised Body Part Segmentation with Pose based Part Priors
Zhengyuan Yang, Yuncheng Li, Linjie Yang +2
Human body part segmentation refers to the task of predicting the semantic segmentation mask for each body part. Fully supervised body part segmentation methods achieve good perfor…
Human-Centered Emotion Recognition in Animated GIFs
Zhengyuan Yang, Yixuan Zhang, Jiebo Luo
As an intuitive way of expression emotion, the animated Graphical Interchange Format (GIF) images have been widely used on social media. Most previous studies on automated GIF emot…
Attentive Relational Networks for Mapping Images to Scene Graphs
Mengshi Qi, Weijian Li, Zhengyuan Yang +2
Scene graph generation refers to the task of automatically mapping an image into a semantic structural graph, which requires correctly labeling each extracted object and their inte…
End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception
Zhengyuan Yang, Yixuan Zhang, Jerry Yu +2
Convolutional Neural Networks (CNN) have been successfully applied to autonomous driving tasks, many in an end-to-end manner. Previous end-to-end steering control methods take an i…
Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences
Zhengyuan Yang, Yuncheng Li, Jianchao Yang +1
Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have show…