665 citations · 4.5k across the 236 of their papers we have counts for
52 papers · 1 filter
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption
Zhengyuan Yang, Yijuan Lu, Jianfeng Wang +6
In this paper, we propose Text-Aware Pre-training (TAP) for Text-VQA and Text-Caption tasks. These two tasks aim at reading and understanding scene text in images for question answ…
Semantic Layout Manipulation with High-Resolution Sparse Attention
Haitian Zheng, Zhe Lin, Jingwan Lu +4
We tackle the problem of semantic image layout manipulation, which aims to manipulate an input image by editing its semantic label map. A core problem of this task is how to transf…
Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language
Songyang Zhang, Houwen Peng, Jianlong Fu +2
We address the problem of retrieving a specific moment from an untrimmed video by natural language. It is a challenging problem because a target moment may take place in the contex…
XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors
Cheng Peng, Haofu Liao, Gina Wong +3
A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane. Hence, radiograph analysis naturally requires phy…
Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination
Hanjia Lyu, Junda Wang, Wei Wu +4
The current development of vaccines for SARS-CoV-2 is unprecedented. Little is known, however, about the nuanced public opinions on the vaccines on social media. We adopt a human-g…
Face Off: Polarized Public Opinions on Personal Face Mask Usage during the COVID-19 Pandemic
Neil Yeung, Jonathan Lai, Jiebo Luo
In spite of a growing body of scientific evidence on the effectiveness of individual face mask usage for reducing transmission rates, individual face mask usage has become a highly…
Content-based Analysis of the Cultural Differences between TikTok and Douyin
Li Sun, Haoqi Zhang, Songyang Zhang +1
Short-form video social media shifts away from the traditional media paradigm by telling the audience a dynamic story to attract their attention. In particular, different combinati…
Slender Object Detection: Diagnoses and Improvements
Zhaoyi Wan, Yimin Chen, Sutao Deng +3
In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely \textbf{slender objects}. In real-world scenarios, slender obj…
Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation
Zhengyuan Yang, Amanda Kay, Yuncheng Li +2
Inspired by the human ability to infer emotions from body language, we propose an automated framework for body language based emotion recognition starting from regular RGB videos.…
Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data
Weijian Li, Wei Zhu, E. Ray Dorsey +1
Parkinsons Disease is a neurological disorder and prevalent in elderly people. Traditional ways to diagnose the disease rely on in-person subjective clinical evaluations on the qua…
Understanding the Hoarding Behaviors during the COVID-19 Pandemic using Large Scale Social Media Data
Xupin Zhang, Hanjia Lyu, Jiebo Luo
The COVID-19 pandemic has affected people's lives around the world on an unprecedented scale. We intend to investigate hoarding behaviors in response to the pandemic using large-sc…
How Political is the Spread of COVID-19 in the United States? An Analysis using Transportation and Weather Data
Karan Vombatkere, Hanjia Lyu, Jiebo Luo
We investigate the difference in the spread of COVID-19 between the states won by Donald Trump (Red) and the states won by Hillary Clinton (Blue) in the 2016 presidential election,…
Region Comparison Network for Interpretable Few-shot Image Classification
Zhiyu Xue, Lixin Duan, Wen Li +2
While deep learning has been successfully applied to many real-world computer vision tasks, training robust classifiers usually requires a large amount of well-labeled data. Howeve…
Dynamic Context-guided Capsule Network for Multimodal Machine Translation
Huan Lin, Fandong Meng, Jinsong Su +5
Multimodal machine translation (MMT), which mainly focuses on enhancing text-only translation with visual features, has attracted considerable attention from both computer vision a…
Learning to Localize Actions from Moments
Fuchen Long, Ting Yao, Zhaofan Qiu +3
With the knowledge of action moments (i.e., trimmed video clips that each contains an action instance), humans could routinely localize an action temporally in an untrimmed video.…
A Smartphone-based System for Real-time Early Childhood Caries Diagnosis
Yipeng Zhang, Haofu Liao, Jin Xiao +3
Early childhood caries (ECC) is the most common, yet preventable chronic disease in children under the age of 6. Treatments on severe ECC are extremely expensive and unaffordable f…
Improving One-stage Visual Grounding by Recursive Sub-query Construction
Zhengyuan Yang, Tianlang Chen, Liwei Wang +1
We improve one-stage visual grounding by addressing current limitations on grounding long and complex queries. Existing one-stage methods encode the entire language query as a sing…
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification
Mang Ye, Jianbing Shen, David J. Crandall +2
Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Due to the large intra-class variations and cross-modality discrep…
A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation
Yongjing Yin, Fandong Meng, Jinsong Su +4
Multi-modal neural machine translation (NMT) aims to translate source sentences into a target language paired with images. However, dominant multi-modal NMT models do not fully exp…
Universal Model for Multi-Domain Medical Image Retrieval
Yang Feng, Yubao Liu, Jiebo Luo
Medical Image Retrieval (MIR) helps doctors quickly find similar patients' data, which can considerably aid the diagnosis process. MIR is becoming increasingly helpful due to the w…
Task-agnostic Temporally Consistent Facial Video Editing
Meng Cao, Haozhi Huang, Hao Wang +6
Recent research has witnessed the advances in facial image editing tasks. For video editing, however, previous methods either simply apply transformations frame by frame or utilize…
Monitoring Depression Trend on Twitter during the COVID-19 Pandemic
Yipeng Zhang, Hanjia Lyu, Yubao Liu +3
The COVID-19 pandemic has severely affected people's daily lives and caused tremendous economic loss worldwide. However, its influence on people's mental health conditions has not…
Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
Jie An, Tao Li, Haozhi Huang +6
Extracting effective deep features to represent content and style information is the key to universal style transfer. Most existing algorithms use VGG19 as the feature extractor, w…
Global Image Sentiment Transfer
Jie An, Tianlang Chen, Songyang Zhang +1
Transferring the sentiment of an image is an unexplored research topic in the area of computer vision. This work proposes a novel framework consisting of a reference image retrieva…