613 citations · 1.5k across the 47 of their papers we have counts for
49 papers
EgoGapBench: Benchmarking Egocentric Action Selection in Multi-Agent Scenes
Jihyeok Jung, Jeewu Lee, Sanghyeop Kim +2
Existing egocentric benchmarks have primarily constructed the egocentric setting from first-person-view data, which makes it difficult to evaluate egocentric perspective itself in…
Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
Haritz Puerto, Martin Gubri, Sangdoo Yun +1
Membership inference attacks (MIA) attempt to verify the membership of a given data sample in the training set for a model. MIA has become relevant in recent years, following the r…
Calibrating Large Language Models Using Their Generations Only
Dennis Ulmer, Martin Gubri, Hwaran Lee +2
As large language models (LLMs) are increasingly deployed in user-facing applications, building trust and maintaining safety by accurately quantifying a model's confidence in its p…
Do Deep Neural Network Solutions Form a Star Domain?
Ankit Sonthalia, Alexander Rubinstein, Ehsan Abbasnejad +1
It has recently been conjectured that neural network solution sets reachable via stochastic gradient descent (SGD) are convex, considering permutation invariances (Entezari et al.,…
Pretrained Visual Uncertainties
Michael Kirchhof, Mark Collier, Seong Joon Oh +1
Accurate uncertainty estimation is vital to trustworthy machine learning, yet uncertainties typically have to be learned for each task anew. This work introduces the first pretrain…
Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks
Bálint Mucsányi, Michael Kirchhof, Seong Joon Oh
Uncertainty quantification, once a singular task, has evolved into a spectrum of tasks, including abstained prediction, out-of-distribution detection, and aleatoric uncertainty qua…
TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification
Martin Gubri, Dennis Ulmer, Hwaran Lee +2
Large Language Model (LLM) services and models often come with legal rules on who can use them and how they must use them. Assessing the compliance of the released LLMs is crucial,…
Exploring Practitioner Perspectives On Training Data Attribution Explanations
Elisa Nguyen, Evgenii Kortukov, Jean Y. Song +1
Explainable AI (XAI) aims to provide insight into opaque model reasoning to humans and as such is an interdisciplinary field by nature. In this paper, we interviewed 10 practitione…
Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles
Luca Scimeca, Alexander Rubinstein, Damien Teney +2
Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut learning, where a model relies on erroneous…
Trustworthy Machine Learning
Bálint Mucsányi, Michael Kirchhof, Elisa Nguyen +2
As machine learning technology gets applied to actual products and solutions, new challenges have emerged. Models unexpectedly fail to generalize to small changes in the distributi…
ProPILE: Probing Privacy Leakage in Large Language Models
Siwon Kim, Sangdoo Yun, Hwaran Lee +3
The rapid advancement and widespread use of large language models (LLMs) have raised significant concerns regarding the potential leakage of personally identifiable information (PI…
URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates
Michael Kirchhof, Bálint Mucsányi, Seong Joon Oh +1
Representation learning has significantly driven the field to develop pretrained models that can act as a valuable starting point when transferring to new datasets. With the rising…
A Bayesian Approach To Analysing Training Data Attribution In Deep Learning
Elisa Nguyen, Minjoon Seo, Seong Joon Oh
Training data attribution (TDA) techniques find influential training data for the model's prediction on the test data of interest. They approximate the impact of down- or up-weight…
Playing repeated games with Large Language Models
Elif Akata, Lion Schulz, Julian Coda-Forno +3
LLMs are increasingly used in applications where they interact with humans and other agents. We propose to use behavioural game theory to study LLM's cooperation and coordination b…
Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts
Dongyoon Han, Junsuk Choe, Seonghyeok Chun +5
Supervised learning of image classifiers distills human knowledge into a parametric model through pairs of images and corresponding labels (X,Y). We argue that this simple and wide…
Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs
Michael Kirchhof, Enkelejda Kasneci, Seong Joon Oh
Contrastively trained encoders have recently been proven to invert the data-generating process: they encode each input, e.g., an image, into the true latent vector that generated t…
SelecMix: Debiased Learning by Contradicting-pair Sampling
Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak +4
Neural networks trained with ERM (empirical risk minimization) sometimes learn unintended decision rules, in particular when their training data is biased, i.e., when training labe…
Scratching Visual Transformer's Back with Uniform Attention
Nam Hyeon-Woo, Kim Yu-Ji, Byeongho Heo +3
The favorable performance of Vision Transformers (ViTs) is often attributed to the multi-head self-attention (MSA). The MSA enables global interactions at each layer of a ViT model…
ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets
Damien Teney, Yong Lin, Seong Joon Oh +1
Several studies have compared the in-distribution (ID) and out-of-distribution (OOD) performance of models in computer vision and NLP. They report a frequent positive correlation a…
Dataset Condensation via Efficient Synthetic-Data Parameterization
Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh +5
The great success of machine learning with massive amounts of data comes at a price of huge computation costs and storage for training and tuning. Recent studies on dataset condens…
ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCO
Sanghyuk Chun, Wonjae Kim, Song Park +2
Image-Text matching (ITM) is a common task for evaluating the quality of Vision and Language (VL) models. However, existing ITM benchmarks have a significant limitation. They have…
Weakly Supervised Semantic Segmentation using Out-of-Distribution Data
Jungbeom Lee, Seong Joon Oh, Sangdoo Yun +3
Weakly supervised semantic segmentation (WSSS) methods are often built on pixel-level localization maps obtained from a classifier. However, training on class labels only, classifi…
ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classification
Hazel Kim, Daecheol Woo, Seong Joon Oh +2
Data augmentation has been an important ingredient for boosting performances of learned models. Prior data augmentation methods for few-shot text classification have led to great p…
Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective
Luca Scimeca, Seong Joon Oh, Sanghyuk Chun +2
Deep neural networks (DNNs) often rely on easy-to-learn discriminatory features, or cues, that are not necessarily essential to the problem at hand. For example, ducks in an image…