5 papers · 1 filter
SoS: Analysis of Surface over Semantics in Multilingual Text-To-Image Generation
Carolin Holtermann, Florian Schneider, Anne Lauscher
Text-to-image (T2I) models are increasingly employed by users worldwide. However, prior research has pointed to the high sensitivity of T2I towards particular input languages - whe…
GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking
Florian Schneider, Carolin Holtermann, Chris Biemann +1
Large Vision-Language Models (LVLMs) have recently gained attention due to their distinctive performance and broad applicability. While it has been previously shown that their effi…
MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching
Fabian David Schmidt, Florian Schneider, Chris Biemann +1
Existing multilingual vision-language (VL) benchmarks often only cover a handful of languages. Consequently, evaluations of large vision-language models (LVLMs) predominantly targe…
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
Gregor Geigle, Florian Schneider, Carolin Holtermann +4
Most Large Vision-Language Models (LVLMs) to date are trained predominantly on English data, which makes them struggle to understand non-English input and fail to generate output i…
M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks
Florian Schneider, Sunayana Sitaram
Since the release of ChatGPT, the field of Natural Language Processing has experienced rapid advancements, particularly in Large Language Models (LLMs) and their multimodal counter…