3 citations · 7 across the 9 of their papers we have counts for
5 papers · 1 filter
Stealing User Prompts from Mixture of Experts
Itay Yona, Ilia Shumailov, Jamie Hayes +1
Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense language models by routing each token to a small number of experts in each layer. In this paper, we…
Measuring memorization in language models via probabilistic extraction
Jamie Hayes, Marika Swanberg, Harsh Chaudhari +6
Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable…
Operationalizing Contextual Integrity in Privacy-Conscious Assistants
Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi +10
Advanced AI assistants combine frontier LLMs and tool access to autonomously perform complex tasks on behalf of users. While the helpfulness of such assistants can increase dramati…
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
Ilia Shumailov, Jamie Hayes, Eleni Triantafillou +6
Exact unlearning was first introduced as a privacy mechanism that allowed a user to retract their data from machine learning models on request. Shortly after, inexact schemes were…
Buffer Overflow in Mixture of Experts
Jamie Hayes, Ilia Shumailov, Itay Yona
Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady. We show that expert routing strategies that have cros…