46 citations · 279 across the 39 of their papers we have counts for
37 papers · 1 filter
The Devil in the Details: Simple and Effective Optical Flow Synthetic Data Generation
Kwon Byung-Ki, Kim Sung-Bin, Tae-Hyun Oh
Recent work on dense optical flow has shown significant progress, primarily in a supervised learning manner requiring a large amount of labeled data. Due to the expensiveness of ob…
TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation
Moon Ye-Bin, Jisoo Kim, Hongyeob Kim +2
We propose TextManiA, a text-driven manifold augmentation method that semantically enriches visual feature spaces, regardless of class distribution. TextManiA augments visual data…
Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment
Kim Sung-Bin, Arda Senocak, Hyunwoo Ha +2
How does audio describe the world around us? In this paper, we propose a method for generating an image of a scene from sound. Our method addresses the challenges of dealing with t…
ENInst: Enhancing Weakly-supervised Low-shot Instance Segmentation
Moon Ye-Bin, Dongmin Choi, Yongjin Kwon +2
We address a weakly-supervised low-shot instance segmentation, an annotation-efficient training method to deal with novel classes effectively. Since it is an under-explored problem…
Semi-Supervised Image Captioning by Adversarially Propagating Labeled Data
Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi +1
We present a novel data-efficient semi-supervised framework to improve the generalization of image captioning models. Constructing a large-scale labeled image captioning dataset is…
Scratching Visual Transformer's Back with Uniform Attention
Nam Hyeon-Woo, Kim Yu-Ji, Byeongho Heo +3
The favorable performance of Vision Transformers (ViTs) is often attributed to the multi-head self-attention (MSA). The MSA enables global interactions at each layer of a ViT model…
HDR-Plenoxels: Self-Calibrating High Dynamic Range Radiance Fields
Kim Jun-Seong, Kim Yu-Ji, Moon Ye-Bin +1
We propose high dynamic range (HDR) radiance fields, HDR-Plenoxels, that learn a plenoptic function of 3D HDR radiance fields, geometry information, and varying camera settings inh…
Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers
Junhyeong Cho, Kim Youwang, Tae-Hyun Oh
Transformer encoder architectures have recently achieved state-of-the-art results on monocular 3D human mesh reconstruction, but they require a substantial number of parameters and…
CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes
Kim Youwang, Kim Ji-Yeon, Tae-Hyun Oh
We propose CLIP-Actor, a text-driven motion recommendation and neural mesh stylization system for human mesh animation. CLIP-Actor animates a 3D human mesh to conform to a text pro…
Audio-Visual Fusion Layers for Event Type Aware Video Recognition
Arda Senocak, Junsik Kim, Tae-Hyun Oh +3
Human brain is continuously inundated with the multisensory information and their complex interactions coming from the outside world at any given moment. Such information is automa…
Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise
Kim Youwang, Kim Ji-Yeon, Kyungdon Joo +1
We propose an end-to-end unified 3D mesh recovery of humans and quadruped animals trained in a weakly-supervised way. Unlike recent work focusing on a single target class only, we…
FICGAN: Facial Identity Controllable GAN for De-identification
Yonghyun Jeong, Jooyoung Choi, Sungwon Kim +5
In this work, we present Facial Identity Controllable GAN (FICGAN) for not only generating high-quality de-identified face images with ensured privacy protection, but also detailed…
Dense Relational Image Captioning via Multi-task Triple-Stream Networks
Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi +1
We introduce dense relational captioning, a novel image captioning task which aims to generate multiple captions with respect to relational information between objects in a visual…
Monocular Reconstruction of Neural Face Reflectance Fields
Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh +7
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing. Most…
Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh +3
Existing unsupervised domain adaptation methods aim to transfer knowledge from a label-rich source domain to an unlabeled target domain. However, obtaining labels for some source d…
Listen to Look: Action Recognition by Previewing Audio
Ruohan Gao, Tae-Hyun Oh, Kristen Grauman +1
In the face of the video data deluge, today's expensive clip-level classifiers are increasingly impractical. We propose a framework for efficient action recognition in untrimmed vi…
Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications
Arda Senocak, Tae-Hyun Oh, Junsik Kim +2
Visual events are usually accompanied by sounds in our daily lives. However, can the machines learn to correlate the visual scene and sound, as well as localize the sound source on…
Visuomotor Understanding for Representation Learning of Driving Scenes
Seokju Lee, Junsik Kim, Tae-Hyun Oh +4
Dashboard cameras capture a tremendous amount of driving scene video each day. These videos are purposefully coupled with vehicle sensing data, such as from the speedometer and ine…
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh +1
Constructing an organized dataset comprised of a large number of images and several captions for each image is a laborious task, which requires vast human effort. On the other hand…
Speech2Face: Learning the Face Behind a Voice
Tae-Hyun Oh, Tali Dekel, Changil Kim +4
How much can we infer about a person's looks from the way they speak? In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of…
Variational Prototyping-Encoder: One-Shot Learning with Prototypical Images
Junsik Kim, Tae-Hyun Oh, Seokju Lee +2
In daily life, graphic symbols, such as traffic signs and brand logos, are ubiquitously utilized around us due to its intuitive expression beyond language boundary. We tackle an op…
Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning
Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh +1
Our goal in this work is to train an image captioning model that generates more dense and informative captions. We introduce "relational captioning," a novel image captioning task…
Neural Inverse Knitting: From Images to Manufacturing Instructions
Alexandre Kaspar, Tae-Hyun Oh, Liane Makatura +3
Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a…
Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
Suwon Shon, Tae-Hyun Oh, James Glass
In this paper, we present a multi-modal online person verification system using both speech and visual signals. Inspired by neuroscientific findings on the association of voice and…