6 citations · 9 across the 7 of their papers we have counts for
8 papers
Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs
Rohit Saxena, Aryo Pradipta Gema, Pasquale Minervini
Understanding time from visual representations is a fundamental cognitive skill, yet it remains a challenge for multimodal large language models (MLLMs). In this work, we investiga…
PosterSum: A Multimodal Benchmark for Scientific Poster Summarization
Rohit Saxena, Pasquale Minervini, Frank Keller
Generating accurate and concise textual summaries from multimodal documents is challenging, especially when dealing with visually complex content like scientific posters. We introd…
What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations
Dongqi Liu, Chenxi Whitehouse, Xi Yu +6
Transforming recorded videos into concise and accurate textual summaries is a growing challenge in multimodal learning. This paper introduces VISTA, a dataset specifically designed…
End-to-End Long Document Summarization using Gradient Caching
Rohit Saxena, Hao Tang, Frank Keller
Training transformer-based encoder-decoder models for long document summarization poses a significant challenge due to the quadratic memory consumption during training. Several app…
MovieSum: An Abstractive Summarization Dataset for Movie Screenplays
Rohit Saxena, Frank Keller
Movie screenplay summarization is challenging, as it requires an understanding of long input contexts and various elements unique to movies. Large language models have shown signif…
Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong +13
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates nu…
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models
Giwon Hong, Aryo Pradipta Gema, Rohit Saxena +8
Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate human-like text. However, the…
Select and Summarize: Scene Saliency for Movie Script Summarization
Rohit Saxena, Frank Keller
Abstractive summarization for long-form narrative texts such as movie scripts is challenging due to the computational and memory constraints of current language models. A movie scr…