5 papers
ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
Hyunseok Lee, Seunghyuk Oh, Jaehyung Kim +2
Self-awareness, i.e., the ability to assess and correct one's own generation, is a fundamental aspect of human intelligence, making its replication in large language models (LLMs)…
Mamba Drafters for Speculative Decoding
Daewon Choi, Seunghyuk Oh, Saket Dingliwal +9
Speculative decoding has emerged as a promising approach to accelerating large language model (LLM) generation using a fast drafter while maintaining alignment with the target mode…
Sparsified State-Space Models are Efficient Highway Networks
Woomin Song, Jihoon Tack, Sangwoo Mo +2
State-space models (SSMs) offer a promising architecture for sequence modeling, providing an alternative to Transformers by replacing expensive self-attention with linear recurrenc…
Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning
Jaehyun Nam, Kyuyoung Kim, Seunghyuk Oh +3
In tabular prediction tasks, tree-based models combined with automated feature engineering methods often outperform deep learning approaches that rely on learned representations. W…
Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
Woomin Song, Seunghyuk Oh, Sangwoo Mo +4
Large language models (LLMs) have shown remarkable performance in various natural language processing tasks. However, a primary constraint they face is the context limit, i.e., the…