6 papers
Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects
Xiaoyu Luo, Wenrui Yu, Qiongxiu Li +1
Autoregressive language models (ARMs) have been shown to memorize and occasionally reproduce training data verbatim, raising concerns about privacy and copyright liability. Diffusi…
APEX: Probing Neural Networks via Activation Perturbation
Tao Ren, Xiaoyu Luo, Qiongxiu Li
Prior work on probing neural networks primarily relies on input-space analysis or parameter perturbation, both of which face fundamental limitations in accessing structural informa…
Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework
Xiaoyu Luo, Yiyi Chen, Qiongxiu Li +1
Large Language Models (LLMs) have been reported to "leak" Personally Identifiable Information (PII), with successful PII reconstruction often interpreted as evidence of memorizatio…
Shared Path: Unraveling Memorization in Multilingual LLMs through Language Similarities
Xiaoyu Luo, Yiyi Chen, Johannes Bjerva +1
We present the first comprehensive study of Memorization in Multilingual Large Language Models (MLLMs), analyzing 95 languages using models across diverse model scales, architectur…
DeMem: Privacy-Enhanced Robust Adversarial Learning via De-Memorization
Xiaoyu Luo, Qiongxiu Li
Adversarial robustness, the ability of a model to withstand manipulated inputs that cause errors, is essential for ensuring the trustworthiness of machine learning models in real-w…
Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond
Qiongxiu Li, Xiaoyu Luo, Yiyi Chen +1
The role of memorization in machine learning (ML) has garnered significant attention, particularly as modern models are empirically observed to memorize fragments of training data.…