25 citations · 80 across the 27 of their papers we have counts for
29 papers
Results of the 1st Asynchronous CASTLE Challenge at the Joint Egocentric Vision Workshop in Conjunction with CVPR 2026
Luca Rossetto, Werner Bailer, Cathal Gurrin +5
This report summarizes the contributions and results of the 1st Asynchronous CASTLE Challenge at the Joint Egocentric Vision Workshop in conjunction with CVPR 2026.
MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding
Shuai Wang, Wangyuan Ding, Yixian Shen +5
Recent vision-language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surface content but struggle with for…
A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
Shuai Wang, Hongyi Zhu, Jia-Hong Huang +6
Understanding artworks requires multi-step reasoning over visual content and cultural, historical, and stylistic context. While recent multimodal large language models show promise…
A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech
Jia-Hong Huang, Seulgi Kim, Yi Chieh Liu +5
Recent diffusion-based text-to-speech (TTS) models achieve high naturalness and expressiveness, yet often suffer from speaker drift, a subtle, gradual shift in perceived speaker id…
VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings
Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2
Real-world multimodal knowledge graphs (MKGs) are inherently heterogeneous, modeling entities that are associated with diverse modalities. Traditional knowledge graph embedding (KG…
SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation
Chengxi Zeng, Yuxuan Jiang, Ge Gao +6
Vision-language segmentation models such as SAM3 enable flexible, prompt-driven visual grounding, but inherit large, general-purpose text encoders originally designed for open-ende…
Analyzing Sustainability Messaging in Large-Scale Corporate Social Media
Ujjwal Sharma, Stevan Rudinac, Ana Mićković +2
In this work, we introduce a multimodal analysis pipeline that leverages large foundation models in vision and language to analyze corporate social media content, with a focus on s…
Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding
Gowreesh Mago, Pascal Mettes, Stevan Rudinac
The automatic understanding of video content is advancing rapidly. Empowered by deeper neural networks and large datasets, machines are increasingly capable of understanding what i…
ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding
Shuai Wang, Ivona Najdenkoska, Hongyi Zhu +4
Understanding visual art requires reasoning across multiple perspectives -- cultural, historical, and stylistic -- beyond mere object recognition. While recent multimodal large lan…
The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding
Luca Rossetto, Werner Bailer, Duc-Tien Dang-Nguyen +11
Egocentric video has seen increased interest in recent years, as it is used in a range of areas. However, most existing datasets are limited to a single perspective. In this paper,…
Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
Jia-Hong Huang, Yixian Shen, Hongyi Zhu +2
Large Language Models (LLMs) have shown remarkable performance across various tasks, but the escalating demands on computational resources pose significant challenges, particularly…
Non-Progressive Influence Maximization in Dynamic Social Networks
Yunming Hui, Shihan Wang, Melisachew Wudage Chekol +2
The influence maximization (IM) problem involves identifying a set of key individuals in a social network who can maximize the spread of influence through their network connections…
Image2Text2Image: A Novel Framework for Label-Free Evaluation of Image-to-Text Generation with Text-to-Image Diffusion Models
Jia-Hong Huang, Hongyi Zhu, Yixian Shen +2
Evaluating the quality of automatically generated image descriptions is a complex task that requires metrics capturing various dimensions, such as grammaticality, coverage, accurac…
Domain-Informed Negative Sampling Strategies for Dynamic Graph Embedding in Meme Stock-Related Social Networks
Yunming Hui, Inez Maria Zwetsloot, Simon Trimborn +1
Social network platforms like Reddit are increasingly impacting real-world economics. Meme stocks are a recent phenomena where price movements are driven by retail investors organi…
A Novel Evaluation Framework for Image2Text Generation
Jia-Hong Huang, Hongyi Zhu, Yixian Shen +3
Evaluating the quality of automatically generated image descriptions is challenging, requiring metrics that capture various aspects such as grammaticality, coverage, correctness, a…
Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning
Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2
In many real-world applications, modeling both the internal structure of sets and their temporal relationships is essential for capturing complex underlying patterns. Sequential mu…
Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks
Shuai Wang, David W. Zhang, Jia-Hong Huang +4
Hypergraphs serve as an effective model for depicting complex connections in various real-world scenarios, from social to biological networks. The development of Hypergraph Neural…
Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models
Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac +1
Image search stands as a pivotal task in multimedia and computer vision, finding applications across diverse domains, ranging from internet search to medical diagnostics. Conventio…
Prototype-Enhanced Hypergraph Learning for Heterogeneous Information Networks
Shuai Wang, Jiayi Shen, Athanasios Efthymiou +4
The variety and complexity of relations in multimedia data lead to Heterogeneous Information Networks (HINs). Capturing the semantics from such networks requires approaches capable…
Multimodal Temporal Fusion Transformers Are Good Product Demand Forecasters
Maarten Sukel, Stevan Rudinac, Marcel Worring
Multimodal demand forecasting aims at predicting product demand utilizing visual, textual, and contextual information. This paper proposes a method for multimodal product demand fo…
PanorAMS: Automatic Annotation for Detecting Objects in Urban Context
Inske Groenen, Stevan Rudinac, Marcel Worring
Large collections of geo-referenced panoramic images are freely available for cities across the globe, as well as detailed maps with location and meta-data on a great variety of ur…
Adaptive Neural Message Passing for Inductive Learning on Hypergraphs
Devanshu Arya, Deepak K. Gupta, Stevan Rudinac +1
Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes an…
Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings
Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2
We propose ArtSAGENet, a novel multimodal architecture that integrates Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), to jointly learn visual and semantic-b…
HyperSAGE: Generalizing Inductive Representation Learning on Hypergraphs
Devanshu Arya, Deepak K. Gupta, Stevan Rudinac +1
Graphs are the most ubiquitous form of structured data representation used in machine learning. They model, however, only pairwise relations between nodes and are not designed for…