6 papers · 1 filter
Steering Prepositional Phrases in Language Models: A Case of with-headed Adjectival and Adverbial Complements in Gemma-2
Stefan Arnold, René Gröbner
Language Models, when generating prepositional phrases, must often decide for whether their complements functions as an instrumental adjunct (describing the verb adverbially) or an…
Memorization in Language Models through the Lens of Intrinsic Dimension
Stefan Arnold
Language Models (LMs) are prone to memorizing parts of their data during training and unintentionally emitting them at generation time, raising concerns about privacy leakage and d…
Characterizing Stereotypical Bias from Privacy-preserving Pre-Training
Stefan Arnold, Rene Gröbner, Annika Schreiner
Differential Privacy (DP) can be applied to raw text by exploiting the spatial arrangement of words in an embedding space. We investigate the implications of such text privatizatio…
Disentangling the Linguistic Competence of Privacy-Preserving BERT
Stefan Arnold, Nils Kemmerzell, Annika Schreiner
Differential Privacy (DP) has been tailored to address the unique challenges of text-to-text privatization. However, text-to-text privatization is known for degrading the performan…
Guiding Text-to-Text Privatization by Syntax
Stefan Arnold, Dilara Yesilbas, Sven Weinzierl
Metric Differential Privacy is a generalization of differential privacy tailored to address the unique challenges of text-to-text privatization. By adding noise to the representati…
Driving Context into Text-to-Text Privatization
Stefan Arnold, Dilara Yesilbas, Sven Weinzierl
\textit{Metric Differential Privacy} enables text-to-text privatization by adding calibrated noise to the vector of a word derived from an embedding space and projecting this noisy…