102 citations · 154 across the 18 of their papers we have counts for
18 papers
Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
Andrew Lee, Fernanda Viégas, Martin Wattenberg
While researchers are finding concepts represented as linear directions in language models, a bag of linear directions fails to capture relational structure. To better understand t…
Decomposing Query-Key Feature Interactions Using Contrastive Covariances
Andrew Lee, Yonatan Belinkov, Fernanda Viégas +1
Despite the central role of attention heads in Transformers, we lack tools to understand why a model attends to a particular token. To address this, we study the query-key (QK) spa…
Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls
Xiaoyan Bai, Itamar Pres, Yuntian Deng +5
Language models are increasingly capable, yet still fail at a seemingly simple task of multi-digit multiplication. In this work, we study why, by reverse-engineering a model that s…
Chronotome: Real-Time Topic Modeling for Streaming Embedding Spaces
Matte Lim, Catherine Yeh, Martin Wattenberg +2
Many real-world datasets -- from an artist's body of work to a person's social media history -- exhibit meaningful semantic changes over time that are difficult to capture with exi…
Story Ribbons: Reimagining Storyline Visualizations with Large Language Models
Catherine Yeh, Tara Menon, Robin Singh Arya +4
Analyzing literature involves tracking interactions between characters, locations, and themes. Visualization has the potential to facilitate the mapping and analysis of these compl…
What Does it Mean for a Neural Network to Learn a "World Model"?
Kenneth Li, Fernanda Viégas, Martin Wattenberg
We propose a set of precise criteria for saying a neural net learns and uses a "world model." The goal is to give an operational meaning to terms that are often used informally, in…
The Geometry of Self-Verification in a Task-Specific Reasoning Model
Andrew Lee, Lihao Sun, Chris Wendler +2
How do reasoning models verify their own answers? We study this question by training a model using DeepSeek R1's recipe on the CountDown task. We leverage the fact that preference…
When Bad Data Leads to Good Models
Kenneth Li, Yida Chen, Fernanda Viégas +1
In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- an…
Shared Global and Local Geometry of Language Model Embeddings
Andrew Lee, Melanie Weber, Fernanda Viégas +1
Researchers have recently suggested that models share common representations. In our work, we find numerous geometric similarities across the token embeddings of large language mod…
Relational Composition in Neural Networks: A Survey and Call to Action
Martin Wattenberg, Fernanda B. Viégas
Many neural nets appear to represent data as linear combinations of "feature vectors." Algorithms for discovering these vectors have seen impressive recent success. However, we arg…
Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner
Kenneth Li, Yiming Wang, Fernanda Viégas +1
We present an approach called Dialogue Action Tokens (DAT) that adapts language model agents to plan goal-directed dialogues. The core idea is to treat each utterance as an action,…
Designing a Dashboard for Transparency and Control of Conversational AI
Yida Chen, Aoyu Wu, Trevor DePodesta +9
Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially gi…
Measuring and Controlling Instruction (In)Stability in Language Model Dialogs
Kenneth Li, Tianle Liu, Naomi Bashkansky +4
System-prompting is a standard tool for customizing language-model chatbots, enabling them to follow a specific instruction. An implicit assumption in the use of system prompts is…
Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model
Yida Chen, Fernanda Viégas, Martin Wattenberg
Latent diffusion models (LDMs) exhibit an impressive ability to produce realistic images, yet the inner workings of these models remain mysterious. Even when trained purely on imag…
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Kenneth Li, Oam Patel, Fernanda Viégas +2
We introduce Inference-Time Intervention (ITI), a technique designed to enhance the "truthfulness" of large language models (LLMs). ITI operates by shifting model activations durin…
The System Model and the User Model: Exploring AI Dashboard Design
Fernanda Viégas, Martin Wattenberg
This is a speculative essay on interface design and artificial intelligence. Recently there has been a surge of attention to chatbots based on large language models, including wide…
AttentionViz: A Global View of Transformer Attention
Catherine Yeh, Yida Chen, Aoyu Wu +3
Transformer models are revolutionizing machine learning, but their inner workings remain mysterious. In this work, we present a new visualization technique designed to help researc…
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Nur Yildirim, Mahima Pushkarna, Nitesh Goyal +2
Artificial intelligence (AI) presents new challenges for the user experience (UX) of products and services. Recently, practitioner-facing resources and design guidelines have becom…