13 citations · 17 across the 7 of their papers we have counts for
8 papers
Some Tokens Behave like Magnets: Revealing Linguistic Organization in the Layers of Language Models
Andrew Liu, Devan Srinivasan, Gerald Penn
We identify a special group of token vectors inside large language models (LLMs), which we term magnetic vectors, that organize the surrounding tokens by either attracting or repel…
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
Zijian Chen, Xueguang Ma, Shengyao Zhuang +17
Deep-Research agents, which integrate large language models (LLMs) with search tools, have shown success in improving the effectiveness of handling complex queries that require ite…
Relational reasoning and inductive bias in transformers and large language models
Jesse Geerts, Andrew Liu, Stephanie Chan +2
Transformer-based models have demonstrated remarkable reasoning abilities, but the mechanisms underlying relational reasoning remain poorly understood. We investigate how transform…
Flash Invariant Point Attention
Andrew Liu, Axel Elaldi, Nicholas T Franklin +4
Invariant Point Attention (IPA) is a key algorithm for geometry-aware modeling in structural biology, central to many protein and RNA models. However, its quadratic complexity limi…
Bio2Token: All-atom tokenization of any biomolecular structure with Mamba
Andrew Liu, Axel Elaldi, Nathan Russell +1
Efficient encoding and representation of large 3D molecular structures with high fidelity is critical for biomolecular design applications. Despite this, many representation learni…
A Role of Environmental Complexity on Representation Learning in Deep Reinforcement Learning Agents
Andrew Liu, Alla Borisyuk
We developed a simulated environment to train deep reinforcement learning agents on a shortcut usage navigation task, motivated by the Dual Solutions Paradigm test used for human n…
What does the Knowledge Neuron Thesis Have to do with Knowledge?
Jingcheng Niu, Andrew Liu, Zining Zhu +1
We reassess the Knowledge Neuron (KN) Thesis: an interpretation of the mechanism underlying the ability of large language models to recall facts from a training corpus. This nascen…
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Nathaniel Li, Alexander Pan, Anjali Gopal +54
The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and che…