5 papers
When the Same Musical Knowledge Forgets Differently: A Clean Probe of Pathway-Dependent Forgetting
Yu Liu, Zhiwei Yang, Wenxiao Zhang +8
A model can learn that the piano piece Für Elise is calm and reflective by listening to the audio or by reading a text description, but does it matter which route that knowledge t…
ACCORD: Action-Conditioned Contextual Grounding for Language Agents
Lai Jiang, Cheng Qian, Zhenhailong Wang +3
User instructions are often underspecified because humans rely on implicit assumptions about the surrounding environment. For large language model (LLM) agents operating in informa…
RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
Yufeng Du, Phillip Harris, Minyang Tian +5
We identify intrinsic limitations of Rotary Positional Embeddings (RoPE) in Transformer-based long-context language models. Our theoretical analysis abstracts away from the specifi…
Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark
Minhui Zhu, Minyang Tian, Xiaocheng Yang +61
While large language models (LLMs) with reasoning capabilities are progressing rapidly on high-school math competitions and coding, can they reason effectively through complex, ope…
Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
Yufeng Du, Minyang Tian, Srikanth Ronanki +7
Large language models (LLMs) often fail to scale their performance on long-context tasks performance in line with the context lengths they support. This gap is commonly attributed…