111 citations · 257 across the 26 of their papers we have counts for
9 papers · 1 filter
Zero-shot audio captioning with audio-language model guidance and audio context keywords
Leonard Salewski, Stefan Fauth, A. Sophia Koepke +1
Zero-shot audio captioning aims at automatically generating descriptive textual captions for audio content without prior training for this task. Different from speech recognition w…
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language
Leonard Salewski, A. Sophia Koepke, Hendrik P. A. Lensch +1
Converting a model's internals to text can yield human-understandable insights about the model. Inspired by the recent success of training-free approaches for image captioning, we…
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model
Karsten Roth, Lukas Thede, Almut Sophia Koepke +3
Training deep networks requires various design decisions regarding for instance their architecture, data augmentation, or optimization. In this work, we find these training variati…
Video-adverb retrieval with compositional adverb-action embeddings
Thomas Hummel, Otniel-Bogdan Mercea, A. Sophia Koepke +1
Retrieving adverbs that describe an action in a video poses a crucial step towards fine-grained video understanding. We propose a framework for video-to-adverb retrieval (and vice…
Text-to-feature diffusion for audio-visual few-shot learning
Otniel-Bogdan Mercea, Thomas Hummel, A. Sophia Koepke +1
Training deep learning models for video classification from audio-visual data commonly requires immense amounts of labeled training data collected via a costly process. A challengi…
Image-free Classifier Injection for Zero-Shot Classification
Anders Christensen, Massimiliano Mancini, A. Sophia Koepke +2
Zero-shot learning models achieve remarkable results on image classification for samples from classes that were not seen during training. However, such models must be trained from…
Addressing caveats of neural persistence with deep graph persistence
Leander Girrbach, Anders Christensen, Ole Winther +2
Neural Persistence is a prominent measure for quantifying neural network complexity, proposed in the emerging field of topological data analysis in deep learning. In this work, how…
Waffling around for Performance: Visual Classification with Random Words and Broad Concepts
Karsten Roth, Jae Myung Kim, A. Sophia Koepke +3
The visual classification performance of vision-language models such as CLIP has been shown to benefit from additional semantic knowledge from large language models (LLMs) such as…
Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval
Jae Myung Kim, A. Sophia Koepke, Cordelia Schmid +1
Cross-modal retrieval methods are the preferred tool to search databases for the text that best matches a query image and vice versa. However, image-text retrieval models commonly…