11 citations · 35 across the 12 of their papers we have counts for
13 papers
A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation
Jarad Forristal, Niloofar Mireshghallah, Greg Durrett +1
Recent work has shown that energy-based language modeling is an effective framework for controllable text generation because it enables flexible integration of arbitrary discrimina…
URL-BERT: Training Webpage Representations via Social Media Engagements
Ayesha Qamar, Chetan Verma, Ahmed El-Kishky +3
Understanding and representing webpages is crucial to online social networks where users may share and engage with URLs. Common language model (LM) encoders such as BERT can be use…
Membership Inference Attacks against Language Models via Neighbourhood Comparison
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin +3
Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the…
Smaller Language Models are Better Black-box Machine-Generated Text Detectors
Niloofar Mireshghallah, Justus Mattern, Sicun Gao +2
With the advent of fluent generative language models that can produce convincing utterances very similar to those written by humans, distinguishing whether a piece of text is machi…
Non-Parametric Temporal Adaptation for Social Media Topic Classification
Fatemehsadat Mireshghallah, Nikolai Vogler, Junxian He +3
User-generated social media data is constantly changing as new trends influence online discussion and personal information is deleted due to privacy concerns. However, most current…
Memorization in NLP Fine-tuning Methods
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang +2
Large language models are shown to present privacy risks through memorization of training data, and several recent works have studied such risks for the pre-training phase. Little…
Mix and Match: Learning-free Controllable Text Generation using Energy Language Models
Fatemehsadat Mireshghallah, Kartik Goyal, Taylor Berg-Kirkpatrick
Recent work on controlled text generation has either required attribute-based fine-tuning of the base language model (LM), or has restricted the parameterization of the attribute d…
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal +2
The wide adoption and application of Masked language models~(MLMs) on sensitive data (from legal to medical) necessitates a thorough quantitative investigation into their privacy v…
An Unsupervised Masking Objective for Abstractive Multi-Document News Summarization
Nikolai Vogler, Songlin Li, Yujie Xu +2
We show that a simple unsupervised masking objective can approach near supervised performance on abstractive multi-document news summarization. Our method trains a state-of-the-art…
Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription
Nikolai Vogler, Jonathan Parkes Allen, Matthew Thomas Miller +1
We present a self-supervised pre-training approach for learning rich visual language representations for both handwritten and printed historical document transcription. After super…
UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis
Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi +3
Global models are trained to be as generalizable as possible, with user invariance considered desirable since the models are shared across multitudes of users. As such, these model…
Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
Fatemehsadat Mireshghallah, Taylor Berg-Kirkpatrick
Text style can reveal sensitive attributes of the author (e.g. race or age) to the reader, which can, in turn, lead to privacy violations and bias in both human and algorithmic dec…
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa +3
Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy implications when training models on user content such…