2.3k citations · 3.9k across the 43 of their papers we have counts for
47 papers
Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Data
Tarun Kalluri, Jihyeon Lee, Kihyuk Sohn +4
We present a simple and efficient method to leverage emerging text-to-image generative models in creating large-scale synthetic supervision for the task of damage assessment from a…
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction
Kuniaki Saito, Kihyuk Sohn, Chen-Yu Lee +1
Large language models require updates to remain up-to-date or adapt to new domains by fine-tuning them with new documents. One key is memorizing the latest information in a way tha…
Improve Supervised Representation Learning with Masked Image Modeling
Kaifeng Chen, Daniel Salz, Huiwen Chang +3
Training visual embeddings with labeled data supervision has been the de facto setup for representation learning in computer vision. Inspired by recent success of adopting masked i…
Label Budget Allocation in Multi-Task Learning
Ximeng Sun, Kihyuk Sohn, Kate Saenko +2
The cost of labeling data often limits the performance of machine learning systems. In multi-task learning, related tasks provide information to each other and improve overall perf…
Collaborative Score Distillation for Consistent Visual Synthesis
Subin Kim, Kyungmin Lee, June Suk Choi +3
Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting…
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
Chen-Yu Lee, Chun-Liang Li, Hao Zhang +13
The recent advent of self-supervised pre-training techniques has led to a surge in the use of multimodal learning in form document understanding. However, existing approaches that…
StyleDrop: Text-to-Image Generation in Any Style
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee +11
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts. However, ambiguities inherent in natural language and out-of-distributi…
Learning Disentangled Prompts for Compositional Image Synthesis
Kihyuk Sohn, Albert Shaw, Yuan Hao +5
We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to…
Video Probabilistic Diffusion Models in Projected Latent Space
Sihyun Yu, Kihyuk Sohn, Subin Kim +1
Despite the remarkable progress in deep generative models, synthesizing high-resolution and temporally coherent videos still remains a challenge due to their high-dimensionality an…
MaskSketch: Unpaired Structure-guided Masked Image Generation
Dina Bashkirova, Jose Lezama, Kihyuk Sohn +2
Recent conditional image generation methods produce images of remarkable diversity, fidelity and realism. However, the majority of these methods allow conditioning only on labels o…
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang +4
In Composed Image Retrieval (CIR), a user combines a query image with text to describe their intended target. Existing methods rely on supervised learning of CIR models using label…
MAGVIT: Masked Generative Video Transformer
Lijun Yu, Yong Cheng, Kihyuk Sohn +8
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model. We introduce a 3D tokenizer to quantize a video into spat…
SPADE: Semi-supervised Anomaly Detection under Distribution Mismatch
Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li +2
Semi-supervised anomaly detection is a common problem, as often the datasets containing anomalies are partially labeled. We propose a canonical framework: Semi-supervised Pseudo-la…
Visual Prompt Tuning for Generative Transfer Learning
Kihyuk Sohn, Yuan Hao, José Lezama +5
Transferring knowledge from an image synthesis model trained on a large dataset is a promising direction for learning generative image models from various domains efficiently. Whil…
Prefix Conditioning Unifies Language and Label Supervision
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang +4
Image-classification datasets have been used to pretrain image recognition models. Recently, web-scale image-caption datasets have emerged as a source of powerful pretraining alter…
Federated Semi-Supervised Learning with Prototypical Networks
Woojung Kim, Keondo Park, Kihyuk Sohn +2
With the increasing computing power of edge devices, Federated Learning (FL) emerges to enable model training without privacy concerns. The majority of existing studies assume the…
Towards Group Robustness in the presence of Partial Group Labels
Vishnu Suresh Lokhande, Kihyuk Sohn, Jinsung Yoon +3
Learning invariant representations is an important requirement when training machine learning models that are driven by spurious correlations in the datasets. These spurious correl…
Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types
Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li +2
We study anomaly clustering, grouping data into coherent clusters of anomaly types. This is different from anomaly detection that aims to divide anomalies from normal data. Unlike…
Object-aware Contrastive Learning for Debiased Scene Representation
Sangwoo Mo, Hyunwoo Kang, Kihyuk Sohn +2
Contrastive self-supervised learning has shown impressive results in learning visual representations from unlabeled images by enforcing invariance against different data augmentati…
Controlling Neural Networks with Rule Representations
Sungyong Seo, Sercan O. Arik, Jinsung Yoon +3
We propose a novel training method that integrates rules into deep learning, in a way the strengths of the rules are controllable at inference. Deep Neural Networks with Controllab…
Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection
Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li +3
Anomaly detection (AD), separating anomalies from normal data, has many applications across domains, from security to healthcare. While most previous works were shown to be effecti…
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
David Berthelot, Rebecca Roelofs, Kihyuk Sohn +2
We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one…
CutPaste: Self-Supervised Learning for Anomaly Detection and Localization
Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon +1
We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data. To this end, we propose a two-stage…
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Chen Wei, Kihyuk Sohn, Clayton Mellina +2
Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied. While existing semi-supervised learning (SSL) methods are known to perform…