1.9k citations · 3.7k across the 35 of their papers we have counts for
41 papers
NOMAD Projection
Brandon Duderstadt, Zach Nussbaum, Laurens van der Maaten
The rapid adoption of generative AI has driven an explosion in the size of datasets consumed and produced by AI models. Traditional methods for unstructured data visualization, suc…
Law of the Weakest Link: Cross Capabilities of Large Language Models
Ming Zhong, Aston Zhang, Xuewei Wang +14
The development and evaluation of Large Language Models (LLMs) have largely focused on individual capabilities. However, this overlooks the intersection of multiple abilities acros…
The Llama 3 Herd of Models
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri +556
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models th…
A Systematic Study of Bias Amplification
Melissa Hall, Laurens van der Maaten, Laura Gustafson +2
Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data. When a model amplifies bias, it makes certain predictions…
Omnivore: A Single Model for Many Visual Modalities
Rohit Girdhar, Mannat Singh, Nikhila Ravi +3
Prior work has studied different visual modalities in isolation and developed separate architectures for recognition of images, videos, and 3D data. Instead, in this paper, we prop…
Revisiting Weakly Supervised Pre-Training of Visual Perception Models
Mannat Singh, Laura Gustafson, Aaron Adcock +7
Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent st…
EIFFeL: Ensuring Integrity for Federated Learning
Amrita Roy Chowdhury, Chuan Guo, Somesh Jha +1
Federated learning (FL) enables clients to collaborate with a server to train a machine learning model. To ensure privacy, the server performs secure aggregation of updates from th…
Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems
Ruihan Wu, Chuan Guo, Awni Hannun +1
Machine-learning systems such as self-driving cars or virtual assistants are composed of a large number of machine-learning models that recognize image content, transcribe speech,…
Making Paper Reviewing Robust to Bid Manipulation Attacks
Ruihan Wu, Chuan Guo, Felix Wu +3
Most computer science conferences rely on paper bidding to assign reviewers to papers. Although paper bidding enables high-quality assignments in days of unprecedented submission n…
Measuring Data Leakage in Machine-Learning Models with Fisher Information
Awni Hannun, Chuan Guo, Laurens van der Maaten
Machine-learning models contain information about the data they were trained on. This information leaks either through the model itself or through predictions made by the model. Co…
Physical Reasoning Using Dynamics-Aware Models
Eltayeb Ahmed, Anton Bakhtin, Laurens van der Maaten +1
A common approach to solving physical reasoning tasks is to train a value learner on example tasks. A limitation of such an approach is that it requires learning about object dynam…
The Trade-Offs of Private Prediction
Laurens van der Maaten, Awni Hannun
Machine learning models leak information about their training data every time they reveal a prediction. This is problematic when the training data needs to remain private. Private…
Forward Prediction for Physical Reasoning
Rohit Girdhar, Laura Gustafson, Aaron Adcock +1
Physical reasoning requires forward prediction: the ability to forecast what will happen next given some initial world state. We study the performance of state-of-the-art forward-p…
Measuring Dataset Granularity
Yin Cui, Zeqi Gu, Dhruv Mahajan +3
Despite the increasing visibility of fine-grained recognition in our field, "fine-grained'' has thus far lacked a precise definition. In this work, building upon clustering theory,…
SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
Yan Wang, Wei-Lun Chao, Kilian Q. Weinberger +1
Few-shot learners aim to recognize new object classes based on a small number of labeled training examples. To prevent overfitting, state-of-the-art few-shot learners use meta-lear…
Privacy-Preserving Multi-Party Contextual Bandits
Awni Hannun, Brian Knott, Shubho Sengupta +1
Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total…
PHYRE: A New Benchmark for Physical Reasoning
Anton Bakhtin, Laurens van der Maaten, Justin Johnson +2
Understanding and reasoning about physics is an important ability of intelligent agents. We develop the PHYRE benchmark for physical reasoning that contains a set of simple classic…
Does Object Recognition Work for Everyone?
Terrance DeVries, Ishan Misra, Changhan Wang +1
The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset. This dataset contains household items and was designed to have…
Defense Against Adversarial Images using Web-Scale Nearest-Neighbor Search
Abhimanyu Dubey, Laurens van der Maaten, Zeki Yalniz +2
A plethora of recent work has shown that convolutional networks are not robust to adversarial images: images that are created by perturbing a sample from the data distribution as t…
Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
Hexiang Hu, Ishan Misra, Laurens van der Maaten
Providing systems the ability to relate linguistic and visual content is one of the hallmarks of computer vision. Tasks such as text-based image retrieval and image captioning were…
Feature Denoising for Improving Adversarial Robustness
Cihang Xie, Yuxin Wu, Laurens van der Maaten +2
Adversarial attacks to image classification systems present challenges to convolutional networks and opportunities for understanding them. This study suggests that adversarial pert…
Anytime Stereo Image Depth Estimation on Mobile Devices
Yan Wang, Zihang Lai, Gao Huang +4
Many applications of stereo depth estimation in robotics require the generation of accurate disparity maps in real time under significant computational constraints. Current state-o…
Exploring the Limits of Weakly Supervised Pretraining
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan +5
State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet,…
Learning by Asking Questions
Ishan Misra, Ross Girshick, Rob Fergus +3
We introduce an interactive learning framework for the development and testing of intelligent visual systems, called learning-by-asking (LBA). We explore LBA in context of the Visu…