179 citations · 1k across the 84 of their papers we have counts for
84 papers
Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts
Dohyeon Kim, Bedionita Soro, Sung Ju Hwang
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large foundation models. However, most…
Continual Learners are Incremental Model Generalizers
Jaehong Yoon, Sung Ju Hwang, Yue Cao
Motivated by the efficiency and rapid convergence of pre-trained models for solving downstream tasks, this paper extensively studies the impact of Continual Learning (CL) models as…
Phrase Retrieval for Open-Domain Conversational Question Answering with Conversational Dependency Modeling via Contrastive Learning
Soyeong Jeong, Jinheon Baek, Sung Ju Hwang +1
Open-Domain Conversational Question Answering (ODConvQA) aims at answering questions through a multi-turn conversation based on a retriever-reader pipeline, which retrieves passage…
Progressive Fourier Neural Representation for Sequential Video Compilation
Haeyong Kang, Jaehong Yoon, DaHyun Kim +2
Neural Implicit Representation (NIR) has recently gained significant attention due to its remarkable ability to encode complex and high-dimensional data into representation space a…
Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation
Minki Kang, Jin Myung Kwak, Jinheon Baek +1
Language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for a conversation that requires factual knowledge, they are…
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models
Hayeon Lee, Rui Hou, Jongpil Kim +3
Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance. Previous studi…
Meta-prediction Model for Distillation-Aware NAS on Unseen Datasets
Hayeon Lee, Sohyun An, Minseon Kim +1
Distillation-aware Neural Architecture Search (DaNAS) aims to search for an optimal student architecture that obtains the best performance and/or efficiency when distilling the kno…
Direct Fact Retrieval from Knowledge Graphs without Entity Linking
Jinheon Baek, Alham Fikri Aji, Jens Lehmann +1
There has been a surge of interest in utilizing Knowledge Graphs (KGs) for various natural language processing/understanding tasks. The conventional mechanism to retrieve facts in…
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks
Minki Kang, Seanie Lee, Jinheon Baek +2
Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks that require a compound understanding of knowledge. However, deployment of the…
Set-based Neural Network Encoding Without Weight Tying
Bruno Andreis, Soro Bedionita, Philip H. S. Torr +1
We propose a neural network weight encoding method for network property prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network paramet…
Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
Jaewoong Lee, Sangwon Jang, Jaehyeong Jo +5
Token-based masked generative models are gaining popularity for their fast inference time with parallel decoding. While recent token-based approaches achieve competitive performanc…
Forget-free Continual Learning with Soft-Winning SubNetworks
Haeyong Kang, Jaehong Yoon, Sultan Rizky Madjid +2
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which states that competitive smooth (non-binary) subnetworks exist within a dense network in continual learning tasks, we…
Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement
Soyeong Jeong, Jinheon Baek, Sung Ju Hwang +1
Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multi…
Graph Generation with Diffusion Mixture
Jaehyeong Jo, Dongki Kim, Sung Ju Hwang
Generation of graphs is a major challenge for real-world tasks that require understanding the complex nature of their non-Euclidean structures. Although diffusion models have achie…
Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Models
Minki Kang, Dongchan Min, Sung Ju Hwang
There has been a significant progress in Text-To-Speech (TTS) synthesis technology in recent years, thanks to the advancement in neural generative modeling. However, existing metho…
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Youngwan Lee, Jeffrey Willette, Jonghee Kim +2
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers. A representative MIM model, the mas…
Learning Transferable Adversarial Robust Representations via Multi-view Consistency
Minseon Kim, Hyeonjeong Ha, Dong Bok Lee +1
Despite the success on few-shot learning problems, most meta-learned models only focus on achieving good performance on clean examples and thus easily break down when given adversa…
Effective Targeted Attacks for Adversarial Self-Supervised Learning
Minseon Kim, Hyeonjeong Ha, Sooel Son +1
Recently, unsupervised adversarial training (AT) has been highlighted as a means of achieving robustness in models without any label information. Previous studies in unsupervised A…
Language Detoxification with Attribute-Discriminative Latent Space
Jin Myung Kwak, Minseon Kim, Sung Ju Hwang
Transformer-based Language Models (LMs) have achieved impressive results on natural language understanding tasks, but they can also generate toxic text such as insults, threats, an…
Self-Distillation for Further Pre-training of Transformers
Seanie Lee, Minki Kang, Juho Lee +2
Pre-training a large transformer model on a massive amount of unlabeled data and fine-tuning it on labeled datasets for diverse downstream tasks has proven to be a successful strat…
On the Soft-Subnetwork for Few-shot Class Incremental Learning
Haeyong Kang, Jaehong Yoon, Sultan Rizky Hikmawan Madjid +2
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive pe…
Dataset Condensation with Latent Space Knowledge Factorization and Sharing
Hae Beom Lee, Dong Bok Lee, Sung Ju Hwang
In this paper, we introduce a novel approach for systematically solving dataset condensation problem in an efficient manner by exploiting the regularity in a given dataset. Instead…
Personalized Subgraph Federated Learning
Jinheon Baek, Wonyong Jeong, Jiongdao Jin +2
Subgraphs of a larger global graph may be distributed across multiple devices, and only locally accessible due to privacy restrictions, although there may be links between subgraph…
KALA: Knowledge-Augmented Language Model Adaptation
Minki Kang, Jinheon Baek, Sung Ju Hwang
Pre-trained language models (PLMs) have achieved remarkable success on various natural language understanding tasks. Simple fine-tuning of PLMs, on the other hand, might be subopti…