23 citations · 58 across the 22 of their papers we have counts for
25 papers
Personalizing Text-to-Image Generation to Individual Taste
Anne-Sofie Maerten, Juliane Verwiebe, Shyamgopal Karthik +3
Modern text-to-image (T2I) models generate high-fidelity visuals but remain indifferent to individual user preferences. While existing reward models optimize for "average" human ap…
It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models
Anne Harrington, A. Sophia Koepke, Shyamgopal Karthik +2
Contemporary text-to-image models exhibit a surprising degree of mode collapse, as can be seen when sampling several images given the same text prompt. Previous work has attempted…
Solving Spatial Supersensing Without Spatial Supersensing
Vishaal Udandarao, Shyamgopal Karthik, Surabhi S. Nath +3
Cambrian-S aims to take the first steps towards improving video world models with spatial supersensing by introducing (i) two benchmarks, VSI-Super-Recall (VSR) and VSI-Super-Count…
Simplifying Knowledge Transfer in Pretrained Models
Siddharth Jain, Shyamgopal Karthik, Vineet Gandhi
Pretrained models are ubiquitous in the current deep learning landscape, offering strong results on a broad range of tasks. Recent works have shown that models differing in various…
Road Obstacle Video Segmentation
Shyam Nandan Rai, Shyamgopal Karthik, Mariana-Iuliana Georgescu +3
With the growing deployment of autonomous driving agents, the detection and segmentation of road obstacles have become critical to ensure safe autonomous navigation. However, exist…
Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
Luca Eyring, Shyamgopal Karthik, Alexey Dosovitskiy +2
The new paradigm of test-time scaling has yielded remarkable breakthroughs in Large Language Models (LLMs) (e.g. reasoning models) and in generative vision models, allowing models…
A Good CREPE needs more than just Sugar: Investigating Biases in Compositional Vision-Language Benchmarks
Vishaal Udandarao, Mehdi Cherti, Shyamgopal Karthik +3
We investigate 17 benchmarks (e.g. SugarCREPE, VALSE) commonly used for measuring compositional understanding capabilities of vision-language models (VLMs). We scrutinize design ch…
Concept-Guided Interpretability via Neural Chunking
Shuchen Wu, Stephan Alaniz, Shyamgopal Karthik +3
Neural networks are often described as black boxes, reflecting the significant challenge of understanding their internal workings and interactions. We propose a different perspecti…
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot +2
Sparse Autoencoders (SAEs) have recently gained attention as a means to improve the interpretability and steerability of Large Language Models (LLMs), both of which are essential f…
Post-hoc Probabilistic Vision-Language Models
Anton Baumann, Rui Li, Marcus Klasson +5
Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks. For this, VLMs deterministically map image…
Scalable Ranked Preference Optimization for Text-to-Image Generation
Shyamgopal Karthik, Huseyin Coskun, Zeynep Akata +3
Direct Preference Optimization (DPO) has emerged as a powerful approach to align text-to-image (T2I) models with human feedback. Unfortunately, successful application of DPO to T2I…
EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval
Thomas Hummel, Shyamgopal Karthik, Mariana-Iuliana Georgescu +1
In Composed Video Retrieval, a video and a textual description which modifies the video content are provided as inputs to the model. The aim is to retrieve the relevant video with…
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization
Luca Eyring, Shyamgopal Karthik, Karsten Roth +2
Text-to-Image (T2I) models have made significant advancements in recent years, but they still struggle to accurately capture intricate details specified in complex compositional pr…
Vision-by-Language for Training-Free Compositional Image Retrieval
Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini +1
Given an image and a target modification (e.g an image of the Eiffel tower and the text "without people and at night-time"), Compositional Image Retrieval (CIR) aims to retrieve th…
ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models
Uddeshya Upadhyay, Shyamgopal Karthik, Massimiliano Mancini +1
Large-scale vision-language models (VLMs) like CLIP successfully find correspondences between images and text. Through the standard deterministic mapping process, an image or a tex…
If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection
Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini +1
Despite their impressive capabilities, diffusion-based text-to-image (T2I) models can lack faithfulness to the text prompt, where generated images may not contain all the mentioned…
Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification
Kanishk Jain, Shyamgopal Karthik, Vineet Gandhi
We investigate the problem of reducing mistake severity for fine-grained classification. Fine-grained classification can be challenging, mainly due to the requirement of domain exp…
BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks
Uddeshya Upadhyay, Shyamgopal Karthik, Yanbei Chen +2
High-quality calibrated uncertainty estimates are crucial for numerous real-world applications, especially for deep learning-based deployed ML systems. While Bayesian deep learning…
KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning
Shyamgopal Karthik, Massimiliano Mancini, Zeynep Akata
The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images, given only a subset of them during training and no pr…
Learning From Long-Tailed Data With Noisy Labels
Shyamgopal Karthik, Jérome Revaud, Boris Chidlovskii
Class imbalance and noisy labels are the norm rather than the exception in many large-scale classification datasets. Nevertheless, most works in machine learning typically assume b…
Bringing Generalization to Deep Multi-View Pedestrian Detection
Jeet Vora, Swetanjal Dutta, Kanishk Jain +2
Multi-view Detection (MVD) is highly effective for occlusion reasoning in a crowded environment. While recent works using deep learning have made significant advances in the field,…
No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks
Shyamgopal Karthik, Ameya Prabhu, Puneet K. Dokania +1
There has been increasing interest in building deep hierarchy-aware classifiers that aim to quantify and reduce the severity of mistakes, and not just reduce the number of errors.…
ViNet: Pushing the limits of Visual Modality for Audio-Visual Saliency Prediction
Samyak Jain, Pradeep Yarlagadda, Shreyank Jyoti +3
We propose the ViNet architecture for audio-visual saliency prediction. ViNet is a fully convolutional encoder-decoder architecture. The encoder uses visual features from a network…
Simple Unsupervised Multi-Object Tracking
Shyamgopal Karthik, Ameya Prabhu, Vineet Gandhi
Multi-object tracking has seen a lot of progress recently, albeit with substantial annotation costs for developing better and larger labeled datasets. In this work, we remove the n…