57 citations · 218 across the 37 of their papers we have counts for
21 papers · 1 filter
TULIP: Token-length Upgraded CLIP
Ivona Najdenkoska, Mohammad Mahdi Derakhshani, Yuki M. Asano +3
We address the challenge of representing long captions in vision-language models, such as CLIP. By design these models are limited by fixed, absolute positional encodings, restrict…
Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning
Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2
In many real-world applications, modeling both the internal structure of sets and their temporal relationships is essential for capturing complex underlying patterns. Sequential mu…
Conditional Modeling Based Automatic Video Summarization
Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen +2
The aim of video summarization is to shorten videos automatically while retaining the key information necessary to convey the overall story. Video summarization methods mainly rely…
Causal Video Summarizer for Video Exploration
Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen +2
Recently, video summarization has been proposed as a method to help video exploration. However, traditional video summarization models only generate a fixed video summary which is…
Query-based Video Summarization with Pseudo Label Supervision
Jia-Hong Huang, Luka Murn, Marta Mrak +1
Existing datasets for manually labelled query-based video summarization are costly and thus small, limiting the performance of supervised deep video summarization models. Self-supe…
Causalainer: Causal Explainer for Automatic Video Summarization
Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen +2
The goal of video summarization is to automatically shorten videos such that it conveys the overall story without losing relevant information. In many application scenarios, improp…
Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions
Jia-Hong Huang, Modar Alfadly, Bernard Ghanem +1
Deep neural networks have been critical in the task of Visual Question Answering (VQA), with research traditionally focused on improving model accuracy. Recently, however, there ha…
Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models
Tom van Sonsbeek, Mohammad Mahdi Derakhshani, Ivona Najdenkoska +2
Medical Visual Question Answering (VQA) is an important challenge, as it would lead to faster and more accurate diagnoses and treatment decisions. Most existing methods approach it…
Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning
Ivona Najdenkoska, Xiantong Zhen, Marcel Worring
Multimodal few-shot learning is challenging due to the large domain gap between vision and language modalities. Existing methods are trying to communicate visual concepts as prompt…
Association Graph Learning for Multi-Task Classification with Category Shifts
Jiayi Shen, Zehao Xiao, Xiantong Zhen +2
In this paper, we focus on multi-task classification, where related classification tasks share the same label space and are learned simultaneously. In particular, we tackle a new s…
PanorAMS: Automatic Annotation for Detecting Objects in Urban Context
Inske Groenen, Stevan Rudinac, Marcel Worring
Large collections of geo-referenced panoramic images are freely available for cities across the globe, as well as detailed maps with location and meta-data on a great variety of ur…
Inside Out Visual Place Recognition
Sarah Ibrahimi, Nanne van Noord, Tim Alpherts +1
Visual Place Recognition (VPR) is generally concerned with localizing outdoor images. However, localizing indoor scenes that contain part of an outdoor scene can be of large value…
Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"
Jia-Hong Huang, Ting-Wei Wu, Chao-Han Huck Yang +1
Automatically generating medical reports for retinal images is one of the promising ways to help ophthalmologists reduce their workload and improve work efficiency. In this work, w…
Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings
Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2
We propose ArtSAGENet, a novel multimodal architecture that integrates Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), to jointly learn visual and semantic-b…
Contextualized Keyword Representations for Multi-modal Retinal Image Captioning
Jia-Hong Huang, Ting-Wei Wu, Marcel Worring
Medical image captioning automatically generates a medical description to describe the content of a given medical image. A traditional medical image captioning model creates a medi…
GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization
Jia-Hong Huang, Luka Murn, Marta Mrak +1
Traditional video summarization methods generate fixed video representations regardless of user interest. Therefore such methods limit users' expectations in content search and exp…
Detecting CNN-Generated Facial Images in Real-World Scenarios
Nils Hulzebosch, Sarah Ibrahimi, Marcel Worring
Artificial, CNN-generated images are now of such high quality that humans have trouble distinguishing them from real images. Several algorithmic detection methods have been propose…
Assessing the Robustness of Visual Question Answering Models
Jia-Hong Huang, Modar Alfadly, Bernard Ghanem +1
Deep neural networks have been playing an essential role in the task of Visual Question Answering (VQA). Until recently, their accuracy has been the main focus of research. Now the…
4-Connected Shift Residual Networks
Andrew Brown, Pascal Mettes, Marcel Worring
The shift operation was recently introduced as an alternative to spatial convolutions. The operation moves subsets of activations horizontally and/or vertically. Spatial convolutio…
Learning Task Relatedness in Multi-Task Learning for Images in Context
Gjorgji Strezoski, Nanne van Noord, Marcel Worring
Multimedia applications often require concurrent solutions to multiple tasks. These tasks hold clues to each-others solutions, however as these relations can be complex this remain…
Many Task Learning with Task Routing
Gjorgji Strezoski, Nanne van Noord, Marcel Worring
Typical multi-task learning (MTL) methods rely on architectural adjustments and a large trainable parameter set to jointly optimize over several tasks. However, when the number of…