3 citations · 8 across the 23 of their papers we have counts for
12 papers · 1 filter
Seeing the Forest for the Trees: Emergent Agreement from Individual Conversations
Adi Simhi, Yaniv Nikankin, Jonathan Herzig +3
As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi-turn conversations w…
Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning
Shiwen Qin, Cameron Barker, Henry Gouk +4
Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. However, context length alone…
When Can Attention Heads Be Statically Defined?
Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1
Some attention heads learn similar patterns across inputs. Reusing these patterns could reduce training cost by avoiding repeated query-key score computation and softmax. Through c…
Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification
Joshua Ong Jun Leang, Haonan Li, Zheng Zhao +7
Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLMs) being highly adept in usi…
Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation
Joshua Ong Jun Leang, Zheng Zhao, Mihaela Cătălina Stoian +5
Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning tra…
Debiasing Without Protected Attributes: Latent Concept Erasure from Textual Profiles
Shun Shao, Zheng Zhao, Anna Korhonen +2
Most fairness research in NLP assumes direct access to protected attributes such as gender, race, or nationality. In practice, however, such information is often unavailable due to…
Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer
Shun Shao, Binxu Wang, Shay B. Cohen +2
Mechanistic interpretability has made it possible to localize circuits underlying specific behaviors in language models, but existing methods are expensive, model-specific, and dif…
Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge
Mengyu Wang, Xiaoying Zhi, Zhiyi Li +4
While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialize…
MoRFI: Monotonic Sparse Autoencoder Feature Identification
Dimitris Dimakopoulos, Shay B. Cohen, Ioannis Konstas
Large language models (LLMs) acquire most of their factual knowledge during the pre-training stage, through next token prediction. Subsequent stages of post-training often introduc…
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
Adi Simhi, Fazl Barez, Martin Tutek +2
How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in…
Spectral Attention Steering for Prompt Highlighting
Weixian Waylon Li, Yuchen Niu, Yongxin Yang +3
Attention steering is an important technique for controlling model focus, enabling capabilities such as prompt highlighting, where the model prioritises user-specified text. Howeve…
Lost in Space? Vision-Language Models Struggle with Relative Camera Pose Estimation
Ken Deng, Yifu Qiu, Yoni Kasten +2
We study whether vision-language models (VLMs) can solve relative camera pose estimation (RCPE) from image pairs, a direct test of multi-view spatial reasoning. We cast RCPE as a d…