207 citations · 1.1k across the 49 of their papers we have counts for
52 papers
Training Data Reconstruction: Privacy due to Uncertainty?
Christina Runkel, Kanchana Vaishnavi Gandikota, Jonas Geiping +2
Being able to reconstruct training data from the parameters of a neural network is a major privacy concern. Previous works have shown that reconstructing training data, under certa…
Efficiently Dispatching Flash Attention For Partially Filled Attention Masks
Agniv Sharma, Jonas Geiping
Transformers are widely used across various applications, many of which yield sparse or partially filled attention matrices. Examples include attention masks designed to reduce the…
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova +5
Detecting text generated by modern large language models is thought to be hard, as both LLMs and humans can exhibit a wide range of complex behaviors. However, we find that a score…
Object Recognition as Next Token Prediction
Kaiyu Yue, Bor-Chun Chen, Jonas Geiping +3
We present an approach to pose object recognition as next token prediction. The idea is to apply a language decoder that auto-regressively predicts the text tokens from image embed…
A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning
Valeriia Cherepanova, Roman Levin, Gowthami Somepalli +5
Academic tabular benchmarks often contain small sets of curated features. In contrast, data scientists typically collect as many features as possible into their datasets, and even…
A Simple and Efficient Baseline for Data Attribution on Images
Vasu Singla, Pedro Sandoval-Segura, Micah Goldblum +2
Data attribution methods play a crucial role in understanding machine learning models, providing insight into which training data points are most responsible for model outputs duri…
Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey
Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping +3
Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses. However, desp…
NEFTune: Noisy Embeddings Improve Instruction Finetuning
Neel Jain, Ping-yeh Chiang, Yuxin Wen +10
We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training. Standard fi…
Baseline Defenses for Adversarial Attacks Against Aligned Language Models
Neel Jain, Avi Schwarzschild, Yuxin Wen +7
As Large Language Models quickly become ubiquitous, it becomes critical to understand their security vulnerabilities. Recent work shows that text optimizers can produce jailbreakin…
Augmenters at SemEval-2023 Task 1: Enhancing CLIP in Handling Compositionality and Ambiguity for Zero-Shot Visual WSD through Prompt Augmentation and Text-To-Image Diffusion
Jie S. Li, Yow-Ting Shiue, Yong-Siang Shih +1
This paper describes our zero-shot approaches for the Visual Word Sense Disambiguation (VWSD) Task in English. Our preliminary study shows that the simple approach of matching cand…
Seeing in Words: Learning to Classify through Language Bottlenecks
Khalid Saifullah, Yuxin Wen, Jonas Geiping +2
Neural networks for computer vision extract uninterpretable features despite achieving high accuracy on benchmarks. In contrast, humans can explain their predictions using succinct…
Bring Your Own Data! Self-Supervised Evaluation for Large Language Models
Neel Jain, Khalid Saifullah, Yuxin Wen +6
With the rise of Large Language Models (LLMs) and their ubiquitous deployment in diverse domains, measuring language model behavior on realistic data is imperative. For example, a…
On the Exploitability of Instruction Tuning
Manli Shu, Jiongxiao Wang, Chen Zhu +3
Instruction tuning is an effective technique to align large language models (LLMs) with human intents. In this work, we investigate how an adversary can exploit instruction tuning…
On the Reliability of Watermarks for Large Language Models
John Kirchenbauer, Jonas Geiping, Yuxin Wen +7
As LLMs become commonplace, machine-generated text has the potential to flood the internet with spam, social media bots, and valueless content. Watermarking is a simple and effecti…
Understanding and Mitigating Copying in Diffusion Models
Gowthami Somepalli, Vasu Singla, Micah Goldblum +2
Images generated by diffusion models like Stable Diffusion are increasingly widespread. Recent works and even lawsuits have shown that these models are prone to replicating their t…
Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust
Yuxin Wen, John Kirchenbauer, Jonas Geiping +1
Watermarking the outputs of generative models is a crucial technique for tracing copyright and preventing potential harm from AI-generated content. In this paper, we introduce a no…
What Can We Learn from Unlearnable Datasets?
Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping +2
In an era of widespread web scraping, unlearnable dataset methods have the potential to protect data privacy by preventing deep neural networks from generalizing. But in addition t…
JPEG Compressed Images Can Bypass Protections Against AI Editing
Pedro Sandoval-Segura, Jonas Geiping, Tom Goldstein
Recently developed text-to-image diffusion models make it easy to edit or create high-quality images. Their ease of use has raised concerns about the potential for malicious editin…
A Cookbook of Self-Supervised Learning
Randall Balestriero, Mark Ibrahim, Vlad Sobal +16
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art wi…
Universal Guidance for Diffusion Models
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild +4
Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining. In this work…
Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery
Yuxin Wen, Neel Jain, John Kirchenbauer +3
The strength of modern generative models lies in their ability to be controlled through text-based prompts. Typical "hard" prompts are made from interpretable words and tokens, and…
A Watermark for Large Language Models
John Kirchenbauer, Jonas Geiping, Yuxin Wen +3
Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically…
Cramming: Training a Language Model on a Single GPU in One Day
Jonas Geiping, Tom Goldstein
Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of reach for mo…
Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models
Gowthami Somepalli, Vasu Singla, Micah Goldblum +2
Cutting-edge diffusion models produce images with high quality and customizability, enabling them to be used for commercial art and graphic design purposes. But do diffusion models…