325 citations · 636 across the 29 of their papers we have counts for
29 papers
Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances
Zhendong Chu, Ruiyi Zhang, Tong Yu +4
To achieve state-of-the-art performance, one still needs to train NER models on large-scale, high-quality annotated data, an asset that is both costly and time-intensive to accumul…
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu +4
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans. Furthermore, recent instruction-following datasets include images as visu…
AIMS: All-Inclusive Multi-Level Segmentation
Lu Qi, Jason Kuen, Weidong Guo +5
Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-…
Learning the Visualness of Text Using Large Vision-Language Models
Gaurav Verma, Ryan A. Rossi, Christopher Tensmeyer +2
Visual text evokes an image in a person's mind, while non-visual text fails to do so. A method to automatically detect visualness in text will enable text-to-image retrieval and ge…
MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding
Zilong Wang, Jiuxiang Gu, Chris Tensmeyer +5
Document images are a ubiquitous source of data where the text is organized in a complex hierarchical structure ranging from fine granularity (e.g., words), medium granularity (e.g…
Delving into Out-of-Distribution Detection with Vision-Language Representations
Yifei Ming, Ziyang Cai, Jiuxiang Gu +3
Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world. The vast majority of OOD detection methods are driven by a single…
User-Entity Differential Privacy in Learning Natural Language Models
Phung Lai, NhatHai Phan, Tong Sun +4
In this paper, we introduce a novel concept of user-entity differential privacy (UeDP) to provide formal privacy protection simultaneously to both sensitive entities in textual dat…
High-Quality Entity Segmentation
Lu Qi, Jason Kuen, Weidong Guo +5
Dense image segmentation tasks e.g., semantic, panoptic) are useful for image editing, but existing methods can hardly generalize well in an in-the-wild setting where there are unr…
Improving the Reliability for Confidence Estimation
Haoxuan Qu, Yanchao Li, Lin Geng Foo +3
Confidence estimation, a task that aims to evaluate the trustworthiness of the model's prediction output during deployment, has received lots of research attention recently, due to…
Meta Spatio-Temporal Debiasing for Video Scene Graph Generation
Li Xu, Haoxuan Qu, Jason Kuen +2
Video scene graph generation (VidSGG) aims to parse the video content into scene graphs, which involves modeling the spatio-temporal contextual information in the video. However, d…
Unified Pretraining Framework for Document Understanding
Jiuxiang Gu, Jason Kuen, Vlad I. Morariu +5
Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabel…
CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation
Lu Qi, Jason Kuen, Zhe Lin +7
To improve instance-level detection/segmentation performance, existing self-supervised and semi-supervised methods extract either task-unrelated or task-specific training signals f…
LAFITE: Towards Language-Free Training for Text-to-Image Generation
Yufan Zhou, Ruiyi Zhang, Changyou Chen +6
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs. While image samples are often easily accessi…
Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling
Dat Huynh, Jason Kuen, Zhe Lin +2
Open-vocabulary instance segmentation aims at segmenting novel classes without mask annotations. It is an important step toward reducing laborious human supervision. Most existing…
Multi-Scale Aligned Distillation for Low-Resolution Detection
Lu Qi, Jason Kuen, Jiuxiang Gu +5
In instance-level detection tasks (e.g., object detection), reducing input resolution is an easy option to improve runtime efficiency. However, this option traditionally hurts the…
Open-World Entity Segmentation
Lu Qi, Jason Kuen, Yi Wang +5
We introduce a new image segmentation task, called Entity Segmentation (ES), which aims to segment all visual entities (objects and stuffs) in an image without predicting their sem…
SelfDoc: Self-Supervised Document Representation Learning
Peizhao Li, Jiuxiang Gu, Jason Kuen +5
We propose SelfDoc, a task-agnostic pre-training framework for document image understanding. Because documents are multimodal and are intended for sequential reading, our framework…
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
Mengnan Du, Varun Manjunatha, Rajiv Jain +5
Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to gene…
UNISON: Unpaired Cross-lingual Image Captioning
Jiahui Gao, Yi Zhou, Philip L. H. Yu +2
Image captioning has emerged as an interesting research field in recent years due to its broad application scenarios. The traditional paradigm of image captioning relies on paired…
Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning
Xiangxi Shi, Xu Yang, Jiuxiang Gu +2
Change Captioning is a task that aims to describe the difference between images with natural language. Most existing methods treat this problem as a difference judgment without the…
Watch It Twice: Video Captioning with a Refocused Video Encoder
Xiangxi Shi, Jianfei Cai, Shafiq Joty +1
With the rapid growth of video data and the increasing demands of various applications such as intelligent video search and assistance toward visually-impaired people, video captio…
Scene Graph Generation with External Knowledge and Image Reconstruction
Jiuxiang Gu, Handong Zhao, Zhe Lin +3
Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc. How…
Unpaired Image Captioning via Scene Graph Alignments
Jiuxiang Gu, Shafiq Joty, Jianfei Cai +3
Most of current image captioning models heavily rely on paired image-caption datasets. However, getting large scale image-caption paired data is labor-intensive and time-consuming.…
Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction
Xiangxi Shi, Jianfei Cai, Jiuxiang Gu +1
The explosion of video data on the internet requires effective and efficient technology to generate captions automatically for people who are not able to watch the videos. Despite…