277 citations · 747 across the 24 of their papers we have counts for
24 papers
Dense Text-to-Image Generation with Attention Modulation
Yunji Kim, Jiyoung Lee, Jin-Hwa Kim +2
Existing text-to-image diffusion models struggle to synthesize realistic images given dense captions, where each text prompt provides a detailed description for a specific image re…
Background-aware Moment Detection for Video Moment Retrieval
Minjoon Jung, Youwon Jang, Seongho Choi +3
Video moment retrieval (VMR) identifies a specific moment in an untrimmed video for a given natural language query. This task is prone to suffer the weak alignment problem innate i…
Query-Efficient Black-Box Red Teaming via Bayesian Optimization
Deokjae Lee, JunYeong Lee, Jung-Woo Ha +4
The deployment of large-scale generative models is often restricted by their potential risk of causing harm to users in unpredictable ways. We focus on the problem of black-box red…
Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
Jaewoong Lee, Sangwon Jang, Jaehyeong Jo +5
Token-based masked generative models are gaining popularity for their fast inference time with parallel decoding. While recent token-based approaches achieve competitive performanc…
Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
Junyoung Seo, Wooseok Jang, Min-Seop Kwak +6
Text-to-3D generation has shown rapid progress in recent days with the advent of score distillation, a methodology of using pretrained text-to-2D diffusion models to optimize neura…
Robust Camera Pose Refinement for Multi-Resolution Hash Encoding
Hwan Heo, Taekyung Kim, Jiyoung Lee +4
Multi-resolution hash encoding has recently been proposed to reduce the computational cost of neural renderings, such as NeRF. This method requires accurate camera poses for the ne…
3D-aware Blending with Generative NeRFs
Hyunsu Kim, Gayoung Lee, Yunjey Choi +2
Image blending aims to combine multiple images seamlessly. It remains challenging for existing 2D-based methods, especially when input images are misaligned due to differences in 3…
Semi-Parametric Video-Grounded Text Generation
Sungdong Kim, Jin-Hwa Kim, Jiyoung Lee +1
Efficient video-language modeling should consider the computational cost because of a large, sometimes intractable, number of video frames. Parametric approaches such as the attent…
SelecMix: Debiased Learning by Contradicting-pair Sampling
Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak +4
Neural networks trained with ERM (empirical risk minimization) sometimes learn unintended decision rules, in particular when their training data is biased, i.e., when training labe…
Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval
Minjoon Jung, Seongho Choi, Joochan Kim +2
Video corpus moment retrieval (VCMR) is the task to retrieve the most relevant video moment from a large video corpus using a natural language query. For narrative videos, e.g., dr…
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae +7
There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better infer…
Mutual Information Divergence: A Unified Metric for Multimodal Generative Models
Jin-Hwa Kim, Yunji Kim, Jiyoung Lee +2
Text-to-image generation and image captioning are recently emerged as a new experimental paradigm to assess machine intelligence. They predict continuous quantity accompanied by th…
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training
Gi-Cheon Kang, Sungdong Kim, Jin-Hwa Kim +2
Visual dialog (VisDial) is a task of answering a sequence of questions grounded in an image, using the dialog history as context. Prior work has trained the dialog agents solely on…
ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning
Jaehoon Oh, Sungnyun Kim, Namgyu Ho +3
Cross-domain few-shot learning (CD-FSL), where there are few target samples under extreme differences between source and target domains, has recently attracted huge attention. Rece…
Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty
Jaehoon Oh, Sungnyun Kim, Namgyu Ho +3
Cross-domain few-shot learning (CD-FSL) has drawn increasing attention for handling large differences between the source and target domains--an important concern in real-world scen…
Semi-orthogonal Embedding for Efficient Unsupervised Anomaly Segmentation
Jin-Hwa Kim, Do-Hyeong Kim, Saehoon Yi +1
We present the efficiency of semi-orthogonal embedding for unsupervised anomaly segmentation. The multi-scale features from pre-trained CNNs are recently used for the localized Mah…
Multi-step Estimation for Gradient-based Meta-learning
Jin-Hwa Kim, Junyoung Park, Yongseok Choi
Gradient-based meta-learning approaches have been successful in few-shot learning, transfer learning, and a wide range of other domains. Despite its efficacy and simplicity, the bu…
Multimodal Dual Attention Memory for Video Story Question Answering
Kyung-Min Kim, Seong-Ho Choi, Jin-Hwa Kim +1
We propose a video story question-answering (QA) architecture, Multimodal Dual Attention Memory (MDAM). The key idea is to use a dual attention mechanism with late fusion. MDAM use…
Bilinear Attention Networks
Jin-Hwa Kim, Jaehyun Jun, Byoung-Tak Zhang
Attention networks in multimodal learning provide an efficient way to utilize given visual information selectively. However, the computational cost to learn attention distributions…
Visual Explanations from Hadamard Product in Multimodal Deep Networks
Jin-Hwa Kim, Byoung-Tak Zhang
The visual explanation of learned representation of models helps to understand the fundamentals of learning. The attentional models of previous works used to visualize the attended…
CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen +5
In this work, we propose a goal-driven collaborative task that combines language, perception, and action. Specifically, we develop a Collaborative image-Drawing game between two ag…
Overcoming Catastrophic Forgetting by Incremental Moment Matching
Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun +2
Catastrophic forgetting is a problem of neural networks that loses the information of the first task after training the second task. Here, we propose a method, i.e. incremental mom…
Multimodal Residual Learning for Visual QA
Jin-Hwa Kim, Sang-Woo Lee, Dong-Hyun Kwak +4
Deep neural networks continue to advance the state-of-the-art of image recognition tasks with various methods. However, applications of these methods to multimodality remain limite…
rnn : Recurrent Library for Torch
Nicholas Léonard, Sagar Waghmare, Yang Wang +1
The rnn package provides components for implementing a wide range of Recurrent Neural Networks. It is built withing the framework of the Torch distribution for use with the nn pack…