2.2k citations · 10.8k across the 93 of their papers we have counts for
101 papers
Guide Your Agent with Adaptive Multimodal Rewards
Changyeon Kim, Younggyo Seo, Hao Liu +4
Developing an agent capable of adapting to unseen environments remains a difficult challenge in imitation learning. This work presents Adaptive Return-conditioned Policy (ARP), an…
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
Taehoon Kim, Pyunghwan Ahn, Sangyun Kim +39
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge. This project is designed t…
3D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation
Sungjun Cho, Dae-Woong Jeong, Sung Moon Ko +5
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there ex…
Lightweight feature encoder for wake-up word detection based on self-supervised speech representation
Hyungjun Lim, Younggwan Kim, Kiho Yeom +4
Self-supervised learning method that provides generalized speech representations has recently received increasing attention. Wav2vec 2.0 is the most famous example, showing remarka…
Hierarchical discriminative learning improves visual representations of biomedical microscopy
Cheng Jiang, Xinhai Hou, Akhil Kondepudi +5
Learning high-quality, self-supervised, visual representations is essential to advance the role of computer vision in biomedical microscopy and clinical medicine. Previous work has…
Multimodal Deep Learning
Cem Akkus, Luyang Chu, Vladana Djakovic +14
This book is the result of a seminar in which we reviewed multimodal approaches and attempted to create a solid overview of the field, starting with the current state-of-the-art ap…
Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching
Byoungjip Kim, Sungik Choi, Dasol Hwang +2
Despite surprising performance on zero-shot transfer, pre-training a large-scale multimodal model is often prohibitive as it requires a huge amount of data and computing resources.…
Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
Sungmin Cha, Sungjun Cho, Dasol Hwang +3
Since the recent advent of regulations for data protection (e.g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sen…
Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Program
Tiange Luo, Honglak Lee, Justin Johnson
3D shapes have complementary abstractions from low-level geometry to part-based hierarchies to languages, which convey different levels of information. This paper presents a unifie…
Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders
Jongseong Jang, Daeun Kyung, Seung Hwan Kim +3
Deep neural networks are increasingly used in medical imaging for tasks such as pathological classification, but they face challenges due to the scarcity of high-quality, expert-la…
UniCLIP: Unified Framework for Contrastive Language-Image Pre-training
Janghyeon Lee, Jongsuk Kim, Hyounguk Shon +4
Pre-training vision-language models with contrastive objectives has shown promising results that are both scalable to large uncurated datasets and transferable to many downstream a…
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
Sungjun Cho, Seonwoo Min, Jinwoo Kim +3
To overcome the quadratic cost of self-attention, recent works have proposed various sparse attention modules, most of which fall under one of two groups: 1) sparse attention under…
RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects
Yunseok Jang, Ruben Villegas, Jimei Yang +3
There have been remarkable successes in computer vision with deep learning. While such breakthroughs show robust performance, there have still been many challenges in learning in-d…
Improving Transferability of Representations via Augmentation-Aware Self-Supervision
Hankook Lee, Kibok Lee, Kimin Lee +2
Recent unsupervised representation learning methods have shown to be effective in a range of vision tasks by learning representations invariant to data augmentations such as random…
Rapid Automated Analysis of Skull Base Tumor Specimens Using Intraoperative Optical Imaging and Artificial Intelligence
Cheng Jiang, Abhishek Bhattacharya, Joseph Linzey +19
Background: Accurate diagnosis of skull base tumors is essential for providing personalized surgical treatment strategies. Intraoperative diagnosis can be challenging due to tumor…
A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexity
Seo Taek Kong, Soomin Jeon, Dongbin Na +3
Deep learning (DL) algorithms rely on massive amounts of labeled data. Semi-supervised learning (SSL) and active learning (AL) aim to reduce this label complexity by leveraging unl…
Self-Imitation Learning by Planning
Sha Luo, Hamidreza Kasaei, Lambert Schomaker
Imitation learning (IL) enables robots to acquire skills quickly by transferring expert knowledge, which is widely adopted in reinforcement learning (RL) to initialize exploration.…
Adversarial Environment Generation for Learning to Navigate the Web
Izzeddin Gur, Natasha Jaques, Kevin Malta +3
Learning to autonomously navigate the web is a difficult sequential decision making task. The state and action spaces are large and combinatorial in nature, and websites are dynami…
State Entropy Maximization with Random Encoders for Efficient Exploration
Younggyo Seo, Lili Chen, Jinwoo Shin +3
Recent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL). However, efficient exploration in high-dimensional observ…
Cross-Modal Contrastive Learning for Text-to-Image Generation
Han Zhang, Jing Yu Koh, Jason Baldridge +2
The output of text-to-image synthesis systems should be coherent, clear, photo-realistic scenes with high semantic fidelity to their conditioned text descriptions. Our Cross-Modal…
Evolving Reinforcement Learning Algorithms
John D. Co-Reyes, Yingjie Miao, Daiyi Peng +5
We propose a method for meta-learning reinforcement learning algorithms by searching over the space of computational graphs which compute the loss function for a value-based model-…
Few-shot Sequence Learning with Transformers
Lajanugen Logeswaran, Ann Lee, Myle Ott +3
Few-shot algorithms aim at learning new tasks provided only a handful of training examples. In this work we investigate few-shot learning in the setting where the data points are s…
Text-to-Image Generation Grounded by Fine-Grained User Attention
Jing Yu Koh, Jason Baldridge, Honglak Lee +1
Localized Narratives is a dataset with detailed natural language descriptions of images paired with mouse traces that provide a sparse, fine-grained visual grounding for phrases. W…
Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
Guangxiang Zhu, Minghao Zhang, Honglak Lee +1
Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by…