121 citations · 292 across the 13 of their papers we have counts for
8 papers · 1 filter
Sparsified State-Space Models are Efficient Highway Networks
Woomin Song, Jihoon Tack, Sangwoo Mo +2
State-space models (SSMs) offer a promising architecture for sequence modeling, providing an alternative to Transformers by replacing expensive self-attention with linear recurrenc…
Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
Woomin Song, Seunghyuk Oh, Sangwoo Mo +4
Large language models (LLMs) have shown remarkable performance in various natural language processing tasks. However, a primary constraint they face is the context limit, i.e., the…
Abstract Reasoning via Logic-guided Generation
Sihyun Yu, Sangwoo Mo, Sungsoo Ahn +1
Abstract reasoning, i.e., inferring complicated patterns from given observations, is a central building block of artificial general intelligence. While humans find the answer by ei…
MASKER: Masked Keyword Regularization for Reliable Text Classification
Seung Jun Moon, Sangwoo Mo, Kimin Lee +2
Pre-trained language models have achieved state-of-the-art accuracies on various text classification tasks, e.g., sentiment analysis, natural language inference, and semantic textu…
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
Jihoon Tack, Sangwoo Mo, Jongheon Jeong +1
Novelty detection, i.e., identifying whether a given sample is drawn from outside the training distribution, is essential for reliable machine learning. To this end, there have bee…
Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning
Sejun Park, Jaeho Lee, Sangwoo Mo +1
Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable perform…