291 citations · 625 across the 17 of their papers we have counts for
16 papers · 1 filter
MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition
Alex Andonian, Samuel G Rodriques, Andrew D White +1
Chemical structures appear in patents and the scientific literature as images. For programmatic usage, such as indexing in databases or constructing machine learning model training…
Three ways to improve feature alignment for open vocabulary detection
Relja Arandjelović, Alex Andonian, Arthur Mensch +3
The core problem in zero-shot open vocabulary detection is how to align visual and text features, so that the detector performs well on unseen classes. Previous approaches train th…
Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines
Camilo Fosco, Emilie Josephs, Alex Andonian +1
Deepfakes can fuel online misinformation. As deepfakes get harder to recognize with the naked eye, human users become more reliant on deepfake detection models to help them decide…
Robust Cross-Modal Representation Learning with Progressive Self-Distillation
Alex Andonian, Shixing Chen, Raffay Hamid
The learning objective of vision-language approach of CLIP does not effectively account for the noisy many-to-many correspondences found in web-harvested image captioning datasets,…
Contrastive Feature Loss for Image Prediction
Alex Andonian, Taesung Park, Bryan Russell +3
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result. Yet, this basic functionality remains an open problem. A popular…
The Algonauts Project 2021 Challenge: How the Human Brain Makes Sense of a World in Motion
R. M. Cichy, K. Dwivedi, B. Lahner +8
The sciences of natural and artificial intelligence are fundamentally connected. Brain-inspired human-engineered AI are now the standard for predicting human brain responses during…
Paint by Word
Alex Andonian, Sabrina Osmany, Audrey Cui +4
We investigate the problem of zero-shot semantic image painting. Instead of painting modifications into an image using only concrete colors or a finite set of semantic concepts, we…
VA-RED: Video Adaptive Redundancy Reduction
Bowen Pan, Rameswar Panda, Camilo Fosco +6
Performing inference on deep learning models for videos remains a challenge due to the large amount of computational resources required to achieve robust recognition. An inherent p…
We Have So Much In Common: Modeling Semantic Relational Set Abstractions in Videos
Alex Andonian, Camilo Fosco, Mathew Monfort +4
Identifying common patterns among events is a key ability in human and machine perception, as it underlies intelligent decision making. We propose an approach for learning semantic…
Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding
Mathew Monfort, Bowen Pan, Kandan Ramakrishnan +7
Videos capture events that typically contain multiple sequential, and simultaneous, actions even in the span of only a few seconds. However, most large-scale datasets built to trai…
GANalyze: Toward Visual Definitions of Cognitive Image Properties
Lore Goetschalckx, Alex Andonian, Aude Oliva +1
We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence. These attributes are o…
Cross-view Semantic Segmentation for Sensing Surroundings
Bowen Pan, Jiankai Sun, Ho Yin Tiga Leung +2
Sensing surroundings plays a crucial role in human spatial perception, as it extracts the spatial configuration of objects as well as the free space from the observations. To facil…
The Algonauts Project: A Platform for Communication between the Sciences of Biological and Artificial Intelligence
Radoslaw Martin Cichy, Gemma Roig, Alex Andonian +6
In the last decade, artificial intelligence (AI) models inspired by the brain have made unprecedented progress in performing real-world perceptual tasks like object classification…
Unsupervised Learning from Video with Deep Neural Embeddings
Chengxu Zhuang, Tianwei She, Alex Andonian +2
Because of the rich dynamical structure of videos and their ubiquity in everyday life, it is a natural idea that video data could serve as a powerful unsupervised learning signal f…
Moments in Time Dataset: one million videos for event understanding
Mathew Monfort, Alex Andonian, Bolei Zhou +8
We present the Moments in Time Dataset, a large-scale human-annotated collection of one million short videos corresponding to dynamic events unfolding within three seconds. Modelin…
Temporal Relational Reasoning in Videos
Bolei Zhou, Alex Andonian, Aude Oliva +1
Temporal relational reasoning, the ability to link meaningful transformations of objects or entities over time, is a fundamental property of intelligent species. In this paper, we…