67 citations · 114 across the 12 of their papers we have counts for
12 papers
Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies
Ritwik Gupta, Leah Walker, Rodolfo Corona +5
Current regulations on powerful AI capabilities are narrowly focused on "foundation" or "frontier" models. However, these terms are vague and inconsistently defined, leading to an…
Hyperbolic Learning with Multimodal Large Language Models
Paolo Mandica, Luca Franco, Konstantinos Kallidromitis +2
Hyperbolic embeddings have demonstrated their effectiveness in capturing measures of uncertainty and hierarchical relationships across various deep-learning tasks, including image…
An Introduction to Vision-Language Modeling
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38
Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guid…
ALOHa: A New Measure for Hallucination in Captioning Models
Suzanne Petryk, David M. Chan, Anish Kachinthaya +4
Despite recent advances in multimodal pre-training for visual description, state-of-the-art models still produce captions containing errors, such as hallucinating objects not prese…
CLAIR: Evaluating Image Captions with Large Language Models
David Chan, Suzanne Petryk, Joseph E. Gonzalez +2
The evaluation of machine-generated image captions poses an interesting yet persistent challenge. Effective evaluation measures must consider numerous dimensions of similarity, inc…
Simple Token-Level Confidence Improves Caption Correctness
Suzanne Petryk, Spencer Whitehead, Joseph E. Gonzalez +3
The ability to judge whether a caption correctly describes an image is a critical part of vision-language understanding. However, state-of-the-art models often misinterpret the cor…
Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers
Kevin Miao, Akash Gokul, Raghav Singh +5
Recent trends in self-supervised representation learning have focused on removing inductive biases from training pipelines. However, inductive biases can be useful in settings when…
Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly
Spencer Whitehead, Suzanne Petryk, Vedaad Shakib +4
Machine learning has advanced dramatically, narrowing the accuracy gap to humans in multimodal tasks like visual question answering (VQA). However, while humans can say "I don't kn…
On Guiding Visual Attention with Language Specification
Suzanne Petryk, Lisa Dunlap, Keyan Nasseri +3
While real world challenges typically define visual categories with language words or phrases, most visual classification methods define categories with numerical indices. However,…
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting
Sayna Ebrahimi, Suzanne Petryk, Akash Gokul +4
The goal of continual learning (CL) is to learn a sequence of tasks without suffering from the phenomenon of catastrophic forgetting. Previous work has shown that leveraging memory…
NBDT: Neural-Backed Decision Trees
Alvin Wan, Lisa Dunlap, Daniel Ho +6
Machine learning applications such as finance and medicine demand accurate and justifiable predictions, barring most deep learning methods from use. In response, previous work comb…
Enhancement in surface mobility and quantum transport of Bi_(2-x)Sb_(x)Te_(3-y)Se_(y) topological insulator by controlling the crystal growth conditions
Kyu-Bum Han, Su Kong Chong, Akira Nagaoka +4
Despite numerous studies on three-dimensional topological insulators (3D TIs), the controlled growth of high quality (bulk-insulating and high mobility) TIs remains a challenging s…