39 citations · 77 across the 12 of their papers we have counts for
12 papers
Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionality
Youngtaek Oh, Jae Won Cho, Dong-Jin Kim +2
In this paper, we propose a new method to enhance compositional understanding in pre-trained vision and language models (VLMs) without sacrificing performance in zero-shot multi-mo…
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
Taehoon Kim, Pyunghwan Ahn, Sangyun Kim +39
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge. This project is designed t…
Self-Sufficient Framework for Continuous Sign Language Recognition
Youngjoon Jang, Youngtaek Oh, Jae Won Cho +4
The goal of this work is to develop self-sufficient framework for Continuous Sign Language Recognition (CSLR) that addresses key issues of sign language recognition. These include…
Semi-Supervised Image Captioning by Adversarially Propagating Labeled Data
Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi +1
We present a novel data-efficient semi-supervised framework to improve the generalization of image captioning models. Constructing a large-scale labeled image captioning dataset is…
Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition
Youngjoon Jang, Youngtaek Oh, Jae Won Cho +3
The goal of this work is background-robust continuous sign language recognition. Most existing Continuous Sign Language Recognition (CSLR) benchmarks have fixed backgrounds and are…
Generative Bias for Robust Visual Question Answering
Jae Won Cho, Dong-jin Kim, Hyeonggon Ryu +1
The task of Visual Question Answering (VQA) is known to be plagued by the issue of VQA models exploiting biases within the dataset to make its final prediction. Various previous en…
Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillation
Jae Won Cho, Dong-Jin Kim, Jinsoo Choi +2
In this work, we address the issues of missing modalities that have arisen from the Visual Question Answer-Difference prediction task and find a novel method to solve the task at h…
Dense Relational Image Captioning via Multi-task Triple-Stream Networks
Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi +1
We introduce dense relational captioning, a novel image captioning task which aims to generate multiple captions with respect to relational information between objects in a visual…
Detecting Human-Object Interactions with Action Co-occurrence Priors
Dong-Jin Kim, Xiao Sun, Jinsoo Choi +2
A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-…
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh +1
Constructing an organized dataset comprised of a large number of images and several captions for each image is a laborious task, which requires vast human effort. On the other hand…
Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning
Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh +1
Our goal in this work is to train an image captioning model that generates more dense and informative captions. We introduce "relational captioning," a novel image captioning task…
Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks
Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh +2
Human behavior understanding is arguably one of the most important mid-level components in artificial intelligence. In order to efficiently make use of data, multi-task learning ha…