3 citations · 7 across the 9 of their papers we have counts for
10 papers
Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models
Itay Yona, Dan Barzilay, Michael Karasik +1
How do language models retrieve entity-specific facts from their parameters? We investigate this question by searching for sparse, entity-selective MLP neurons - which we call enti…
In-Context Representation Hijacking
Itay Yona, Amir Sarid, Michael Karasik +1
We introduce , a simple in-context representation hijacking attack against large language models (LLMs). The attack works by systematically replacing a harmfu…
Extracting alignment data in open models
Federico Barbero, Xiangming Gu, Christopher A. Choquette-Choo +6
In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model -- useful to steer the model to improve certain capabi…
Lessons from Defending Gemini Against Indirect Prompt Injections
Chongyang Shi, Sharon Lin, Shuang Song +11
Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require…
Interpreting the Repeated Token Phenomenon in Large Language Models
Itay Yona, Ilia Shumailov, Jamie Hayes +2
Large Language Models (LLMs), despite their impressive capabilities, often fail to accurately repeat a single word when prompted to, and instead output unrelated text. This unexpla…
Stealing User Prompts from Mixture of Experts
Itay Yona, Ilia Shumailov, Jamie Hayes +1
Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense language models by routing each token to a small number of experts in each layer. In this paper, we…
Measuring memorization in language models via probabilistic extraction
Jamie Hayes, Marika Swanberg, Harsh Chaudhari +6
Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable…
Operationalizing Contextual Integrity in Privacy-Conscious Assistants
Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi +10
Advanced AI assistants combine frontier LLMs and tool access to autonomously perform complex tasks on behalf of users. While the helpfulness of such assistants can increase dramati…
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
Ilia Shumailov, Jamie Hayes, Eleni Triantafillou +6
Exact unlearning was first introduced as a privacy mechanism that allowed a user to retract their data from machine learning models on request. Shortly after, inexact schemes were…
Buffer Overflow in Mixture of Experts
Jamie Hayes, Ilia Shumailov, Itay Yona
Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady. We show that expert routing strategies that have cros…