67 citations · 129 across the 22 of their papers we have counts for
27 papers
Fast Reconstruction of Exact Maxwell Dynamics from Sparse Data
Dan DeGenaro, Xin Li, Obed Amo +4
We introduce FLASH-MAX, a shallow, exact-by-construction neural network architecture for predicting homogeneous electromagnetic fields from sparse pointwise observations. Each hidd…
D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition
Rupayan Mallick, Sibo Dong, Nataniel Ruiz +1
Applications of diffusion models for visual tasks have been quite noteworthy. This paper targets making classification models more robust to occlusions for the task of object recog…
FaithFill: Faithful Inpainting for Object Completion Using a Single Reference Image
Rupayan Mallick, Amr Abdalla, Sarah Adel Bargal
We present FaithFill, a diffusion-based inpainting object completion approach for realistic generation of missing object parts. Typically, multiple reference images are needed to a…
Lasagna: Layered Score Distillation for Disentangled Object Relighting
Dina Bashkirova, Arijit Ray, Rupayan Mallick +4
Professional artists, photographers, and other visual content creators use object relighting to establish their photo's desired effect. Unfortunately, manual tools that allow relig…
Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing
Ariel N. Lee, Sarah Adel Bargal, Janavi Kasera +3
Vision transformers (ViTs) have significantly changed the computer vision landscape and have periodically exhibited superior performance in vision tasks compared to convolutional n…
VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting
Dina Bashkirova, Samarth Mishra, Diala Lteif +16
Label-efficient and reliable semantic segmentation is essential for many real-life applications, especially for industrial settings with high visual diversity, such as waste sortin…
Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing
Nataniel Ruiz, Sarah Adel Bargal, Cihang Xie +2
Modern deep neural networks tend to be evaluated on static test sets. One shortcoming of this is the fact that these deep neural networks cannot be easily evaluated for robustness…
Temporal Relevance Analysis for Video Action Models
Quanfu Fan, Donghyun Kim, Chun-Fu +4
In this paper, we provide a deep analysis of temporal modeling for action recognition, an important but underexplored problem in the literature. We first propose a new approach to…
Examining the Human Perceptibility of Black-Box Adversarial Attacks on Face Recognition
Benjamin Spetter-Goldstein, Nataniel Ruiz, Sarah Adel Bargal
The modern open internet contains billions of public images of human faces across the web, especially on social media websites used by half the world's population. In this context,…
Simulated Adversarial Testing of Face Recognition Models
Nataniel Ruiz, Adam Kortylewski, Weichao Qiu +4
Most machine learning models are validated and tested on fixed datasets. This can give an incomplete picture of the capabilities and weaknesses of the model. Such weaknesses can be…
SegNBDT: Visual Decision Rules for Segmentation
Alvin Wan, Daniel Ho, Younjin Song +3
The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks li…
Protecting Against Image Translation Deepfakes by Leaking Universal Perturbations from Black-Box Neural Networks
Nataniel Ruiz, Sarah Adel Bargal, Stan Sclaroff
In this work, we develop efficient disruptions of black-box image translation deepfake generation systems. We are the first to demonstrate black-box deepfake generation disruption…
Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems
Nataniel Ruiz, Sarah Adel Bargal, Stan Sclaroff
Face modification systems using deep learning have become increasingly powerful and accessible. Given images of a person's face, such systems can generate new images of that same p…
NBDT: Neural-Backed Decision Trees
Alvin Wan, Lisa Dunlap, Daniel Ho +6
Machine learning applications such as finance and medicine demand accurate and justifiable predictions, barring most deep learning methods from use. In response, previous work comb…
Explainable Deep Classification Models for Domain Generalization
Andrea Zunino, Sarah Adel Bargal, Riccardo Volpi +5
Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object…
DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition
Nuno C. Garcia, Sarah Adel Bargal, Vitaly Ablavsky +3
In this work, we address the problem of learning an ensemble of specialist networks using multimodal data, while considering the realistic and challenging scenario of possible miss…
Multi-way Encoding for Robustness
Donghyun Kim, Sarah Adel Bargal, Jianming Zhang +1
Deep models are state-of-the-art for many computer vision tasks including image classification and object detection. However, it has been shown that deep models are vulnerable to a…
Guided Zoom: Questioning Network Evidence for Fine-grained Classification
Sarah Adel Bargal, Andrea Zunino, Vitali Petsiuk +4
We propose Guided Zoom, an approach that utilizes spatial grounding of a model's decision to make more informed predictions. It does so by making sure the model has "the right reas…
Excitation Dropout: Encouraging Plasticity in Deep Neural Networks
Andrea Zunino, Sarah Adel Bargal, Pietro Morerio +3
We propose a guided dropout regularizer for deep networks based on the evidence of a network prediction defined as the firing of neurons in specific paths. In this work, we utilize…
Hashing with Mutual Information
Fatih Cakir, Kun He, Sarah Adel Bargal +1
Binary vector embeddings enable fast nearest neighbor retrieval in large databases of high-dimensional objects, and play an important role in many practical applications, such as i…
Moments in Time Dataset: one million videos for event understanding
Mathew Monfort, Alex Andonian, Bolei Zhou +8
We present the Moments in Time Dataset, a large-scale human-annotated collection of one million short videos corresponding to dynamic events unfolding within three seconds. Modelin…
Excitation Backprop for RNNs
Sarah Adel Bargal, Andrea Zunino, Donghyun Kim +3
Deep models are state-of-the-art for many vision tasks including video action recognition and video captioning. Models are trained to caption or classify activity in videos, but li…
Hashing as Tie-Aware Learning to Rank
Kun He, Fatih Cakir, Sarah Adel Bargal +1
Hashing, or learning binary embeddings of data, is frequently used in nearest neighbor retrieval. In this paper, we develop learning to rank formulations for hashing, aimed at dire…
MIHash: Online Hashing with Mutual Information
Fatih Cakir, Kun He, Sarah Adel Bargal +1
Learning-based hashing methods are widely used for nearest neighbor retrieval, and recently, online hashing methods have demonstrated good performance-complexity trade-offs by lear…