9 papers
MacroLens: A Multi-Task Benchmark for Contextual Financial Reasoning under Macroeconomic Scenarios
Patara Trirat, Jin Myung Kwak, Jay Heo +2
Financial decision-making is contextual: forecasting prices, valuing companies, and assessing event exposure weigh price history, accounting fundamentals, macroeconomic regime, and…
OmniRetrieval: Unified Retrieval across Heterogeneous Knowledge Sources
Jinheon Baek, Soyeong Jeong, Sangwoo Park +5
Real-world information needs require access to structurally diverse knowledge sources, from unstructured text and relational tables to knowledge graphs and property graphs. Existin…
Multimodal Collaborative Debate for Zero-Shot Time Series Reasoning
Patara Trirat, Jin Myung Kwak, Jay Heo +2
Large language models (LLMs) are increasingly used as natural-language interfaces to structured data, yet they remain brittle when reasoning over time series. Visual patterns can b…
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Dong Bok Lee, Seanie Lee, Sangwoo Park +12
The reliability of large language models (LLMs) during test-time scaling is often assessed with \emph{external verifiers} or \emph{reward models} that distinguish correct reasoning…
Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction
Jeffrey Willette, Heejun Lee, Sung Ju Hwang
The attention mechanism of a transformer has a quadratic complexity, leading to high inference costs and latency for long sequences. However, attention matrices are mostly sparse,…
InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
Heejun Lee, Geon Park, Jaduk Suh +1
In modern large language models (LLMs), handling very long context lengths presents significant challenges as it causes slower inference speeds and increased memory costs. Addition…
Training-Free Exponential Context Extension via Cascading KV Cache
Jeffrey Willette, Heejun Lee, Youngwan Lee +2
The transformer's context window is vital for tasks such as few-shot learning and conditional generation as it preserves previous tokens for active memory. However, as the context…
A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention
Heejun Lee, Geon Park, Youngwan Lee +7
In modern large language models (LLMs), increasing the context length is crucial for improving comprehension and coherence in long-context, multi-modal, and retrieval-augmented lan…
SEA: Sparse Linear Attention with Estimated Attention Mask
Heejun Lee, Jina Kim, Jeffrey Willette +1
The transformer architecture has driven breakthroughs in recent years on tasks which require modeling pairwise relationships between sequential elements, as is the case in natural…