3 citations · 8 across the 23 of their papers we have counts for
27 papers
Seeing the Forest for the Trees: Emergent Agreement from Individual Conversations
Adi Simhi, Yaniv Nikankin, Jonathan Herzig +3
As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi-turn conversations w…
Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning
Shiwen Qin, Cameron Barker, Henry Gouk +4
Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. However, context length alone…
When Can Attention Heads Be Statically Defined?
Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1
Some attention heads learn similar patterns across inputs. Reusing these patterns could reduce training cost by avoiding repeated query-key score computation and softmax. Through c…
Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification
Joshua Ong Jun Leang, Haonan Li, Zheng Zhao +7
Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLMs) being highly adept in usi…
Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation
Joshua Ong Jun Leang, Zheng Zhao, Mihaela Cătălina Stoian +5
Modern Lean theorem provers achieve strong performance only with substantial training and inference compute, driven in part by scarce verified proof data and the long reasoning tra…
Debiasing Without Protected Attributes: Latent Concept Erasure from Textual Profiles
Shun Shao, Zheng Zhao, Anna Korhonen +2
Most fairness research in NLP assumes direct access to protected attributes such as gender, race, or nationality. In practice, however, such information is often unavailable due to…
Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer
Shun Shao, Binxu Wang, Shay B. Cohen +2
Mechanistic interpretability has made it possible to localize circuits underlying specific behaviors in language models, but existing methods are expensive, model-specific, and dif…
Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge
Mengyu Wang, Xiaoying Zhi, Zhiyi Li +4
While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialize…
MoRFI: Monotonic Sparse Autoencoder Feature Identification
Dimitris Dimakopoulos, Shay B. Cohen, Ioannis Konstas
Large language models (LLMs) acquire most of their factual knowledge during the pre-training stage, through next token prediction. Subsequent stages of post-training often introduc…
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
Adi Simhi, Fazl Barez, Martin Tutek +2
How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in…
Spectral Attention Steering for Prompt Highlighting
Weixian Waylon Li, Yuchen Niu, Yongxin Yang +3
Attention steering is an important technique for controlling model focus, enabling capabilities such as prompt highlighting, where the model prioritises user-specified text. Howeve…
Lost in Space? Vision-Language Models Struggle with Relative Camera Pose Estimation
Ken Deng, Yifu Qiu, Yoni Kasten +2
We study whether vision-language models (VLMs) can solve relative camera pose estimation (RCPE) from image pairs, a direct test of multi-view spatial reasoning. We cast RCPE as a d…
ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architectures
Shiwen Qin, Alexander Auras, Shay B. Cohen +4
Neural architecture search (NAS) automates the design process of high-performing architectures, but remains bottlenecked by expensive performance evaluation. Most existing studies…
PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema +7
Best-of-n sampling improves the accuracy of large language models (LLMs) and large reasoning models (LRMs) by generating multiple candidate solutions and selecting the one with the…
PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants
Zheng Zhao, Clara Vania, Subhradeep Kayal +3
Large language models (LLMs) have advanced conversational AI assistants. However, systematically evaluating how well these assistants apply personalization--adapting to individual…
Iterative Multilingual Spectral Attribute Erasure
Shun Shao, Yftah Ziser, Zheng Zhao +3
Multilingual representations embed words with similar meanings to share a common semantic space across languages, creating opportunities to transfer debiasing effects between langu…
Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics
Yifu Qiu, Yftah Ziser, Anna Korhonen +2
Can unified vision-language models (VLMs) perform forward dynamics prediction (FDP), i.e., predicting the future state (in image form) given the previous observation and an action…
Transferrable Surrogates in Expressive Neural Architecture Search Spaces
Shiwen Qin, Gabriela Kadlecová, Martin Pilát +5
Neural architecture search (NAS) faces a challenge in balancing the exploration of expressive, broad search spaces that enable architectural innovation with the need for efficient…
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
Yifu Qiu, Varun Embar, Yizhe Zhang +3
Recent advancements in long-context language models (LCLMs) promise to transform Retrieval-Augmented Generation (RAG) by simplifying pipelines. With their expanded context windows,…
TSPRank: Bridging Pairwise and Listwise Methods with a Bilinear Travelling Salesman Model
Weixian Waylon Li, Yftah Ziser, Yifei Xie +2
Traditional Learning-To-Rank (LETOR) approaches, including pairwise methods like RankNet and LambdaMART, often fall short by solely focusing on pairwise comparisons, leading to sub…
Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
Zheng Zhao, Yftah Ziser, Shay B. Cohen
Fine-tuning pre-trained large language models (LLMs) on a diverse array of tasks has become a common approach for building models that can solve various natural language processing…
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
Tingchen Fu, Mrinank Sharma, Philip Torr +3
Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks. To address this concern, we introduce PoisonBenc…
einspace: Searching for Neural Architectures from Fundamental Operations
Linus Ericsson, Miguel Espinosa, Chenhongyi Yang +5
Neural architecture search (NAS) finds high performing networks for a given task. Yet the results of NAS are fairly prosaic; they did not e.g. create a shift from convolutional str…
LeanReasoner: Boosting Complex Logical Reasoning with Lean
Dongwei Jiang, Marcio Fonseca, Shay B. Cohen
Large language models (LLMs) often struggle with complex logical reasoning due to logical inconsistencies and the inherent difficulty of such reasoning. We use Lean, a theorem prov…