42 citations · 66 across the 3 of their papers we have counts for
5 papers
On the creation of narrow AI: hierarchy and nonlocality of neural network skills
Eric J. Michaud, Asher Parker-Sartori, Max Tegmark
We study the problem of creating strong, yet narrow, AI systems. While recent AI progress has been driven by the training of large general-purpose foundation models, the creation o…
The Remarkable Robustness of LLMs: Stages of Inference?
Vedang Lad, Jin Hwa Lee, Wes Gurnee +1
We investigate the robustness of Large Language Models (LLMs) to structural interventions by deleting and swapping adjacent layers during inference. Surprisingly, models retain 72-…
Not All Language Model Features Are One-Dimensionally Linear
Joshua Engels, Eric J. Michaud, Isaac Liao +2
Recent work has proposed that language models perform computation by manipulating one-dimensional representations of concepts ("features") in activation space. In contrast, we expl…
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
Samuel Marks, Max Tegmark
Large Language Models (LLMs) have impressive capabilities, but are prone to outputting falsehoods. Recent work has developed techniques for inferring whether a LLM is telling the t…
Language Models Represent Space and Time
Wes Gurnee, Max Tegmark
The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent…