4 papers
The First Drop of Ink: Nonlinear Impact of Distracting Information in Long-Context Reasoning
Muhan Gao, Zih-Ching Chen, Kuan-Hao Huang
As large language models are increasingly deployed in retrieval-augmented generation and agentic systems that accumulate extensive context, understanding how distracting informatio…
From Papers to Panoramas: Building Hierarchies of Scientific Literature at Scale
Muhan Gao, Jash Shah, Weiqi Wang +2
Scientific knowledge is growing rapidly, making it difficult to track progress and high-level conceptual links across broad disciplines. While tools like citation networks and sear…
ICL CIPHERS: Quantifying "Learning" in In-Context Learning via Substitution Ciphers
Zhouxiang Fang, Aayush Mishra, Muhan Gao +2
Recent works have suggested that In-Context Learning (ICL) operates in dual modes, i.e. task retrieval (remember learned patterns from pre-training) and task learning (inference-ti…
Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell
Taiming Lu, Muhan Gao, Kuai Yu +2
Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts. Our study explores LLMs' long-context reasoning by…