75 citations · 372 across the 57 of their papers we have counts for
15 papers · 1 filter
Descriminative-Generative Custom Tokens for Vision-Language Models
Pramuditha Perera, Matthew Trager, Luca Zancato +2
This paper explores the possibility of learning custom tokens for representing new concepts in Vision-Language Models (VLMs). Our aim is to learn tokens that can be effective for b…
Diffusion Soup: Model Merging for Text-to-Image Diffusion Models
Benjamin Biggs, Arjun Seshadri, Yang Zou +6
We present Diffusion Soup, a compartmentalization method for Text-to-Image Generation that averages the weights of diffusion models trained on sharded data. By construction, our ap…
Principles of Designing Robust Remote Face Anti-Spoofing Systems
Xiang Xu, Tianchen Zhao, Zheng Zhang +4
Protecting digital identities of human face from various attack vectors is paramount, and face anti-spoofing plays a crucial role in this endeavor. Current approaches primarily foc…
NeRF-Insert: 3D Local Editing with Multimodal Control Signals
Benet Oriol Sabat, Alessandro Achille, Matthew Trager +1
We propose NeRF-Insert, a NeRF editing framework that allows users to make high-quality local edits with a flexible level of control. Unlike previous work that relied on image-to-i…
Multi-Modal Hallucination Control by Visual Information Grounding
Alessandro Favero, Luca Zancato, Matthew Trager +5
Generative Vision-Language Models (VLMs) are prone to generate plausible-sounding textual answers that, however, are not always grounded in the input image. We investigate this phe…
Interpretable Measures of Conceptual Similarity by Complexity-Constrained Descriptive Auto-Encoding
Alessandro Achille, Greg Ver Steeg, Tian Yu Liu +3
Quantifying the degree of similarity between images is a key copyright issue for image-based machine learning. In legal doctrine however, determining the degree of similarity betwe…
Towards Visual Foundational Models of Physical Scenes
Chethan Parameshwara, Alessandro Achille, Matthew Trager +7
We describe a first step towards learning general-purpose visual representations of physical scenes using only image prediction as a training criterion. To do so, we first define "…
Prompt Algebra for Task Composition
Pramuditha Perera, Matthew Trager, Luca Zancato +2
We investigate whether prompts learned independently for different tasks can be later combined through prompt algebra to obtain a model that supports composition of tasks. We consi…
Train/Test-Time Adaptation with Retrieval
Luca Zancato, Alessandro Achille, Tian Yu Liu +3
We introduce Train/Test-Time Adaptation with Retrieval (), a method to adapt models both at train and test time by means of a retrieval module and a searchable pool of…
A Meta-Learning Approach to Predicting Performance and Data Requirements
Achin Jain, Gurumurthy Swaminathan, Paolo Favaro +8
We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model pe…
Mixed Differential Privacy in Computer Vision
Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang +3
We introduce AdaMix, an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. While pre-training language…
A linearized framework and a new benchmark for model selection for fine-tuning
Aditya Deshpande, Alessandro Achille, Avinash Ravichandran +6
Fine-tuning from a collection of models pre-trained on different domains (a "model zoo") is emerging as a technique to improve test accuracy in the low-data regime. However, model…
Supervised Momentum Contrastive Learning for Few-Shot Classification
Orchid Majumder, Avinash Ravichandran, Subhransu Maji +3
Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the cl…
LayoutTransformer: Layout Generation and Completion with Self-attention
Kamal Gupta, Justin Lazarow, Alessandro Achille +3
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents, and 3D objects. Most complex scenes, natural or human-designed…
TextTubes for Detecting Curved Text in the Wild
Joël Seytre, Jon Wu, Alessandro Achille
We present a detector for curved text in natural images. We model scene text instances as tubes around their medial axes and introduce a parametrization-invariant loss function. We…