1.3k citations · 1.6k across the 15 of their papers we have counts for
17 papers
Game Arena: Strategic LLM Evaluation in Competitive Environments
Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena…
GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labeling
Hritik Bansal, Po-Nien Kung, P. Jeffrey Brantingham +2
Multimodal event argument role labeling (EARL), a task that assigns a role for each event participant (object) in an image is a complex challenge. It requires reasoning over the en…
MemeGraphs: Linking Memes to Knowledge Graphs
Vasiliki Kougia, Simon Fetzel, Thomas Kirchmair +4
Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and s…
Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays
Yiheng Xiong, Jingsong Liu, Kamilia Zaripova +3
The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating th…
Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning
Baran Hashemi, Nikolai Hartmann, Sahand Sharifzadeh +2
Simulating high-resolution detector responses is a computationally intensive process that has long been challenging in Particle Physics. Despite the ability of generative models to…
Do DALL-E and Flamingo Understand Each Other?
Hang Li, Jindong Gu, Rajat Koner +2
The field of multimodal research focusing on the comprehension and creation of both images and text has witnessed significant strides. This progress is exemplified by the emergence…
InstanceFormer: An Online Video Instance Segmentation Framework
Rajat Koner, Tanveer Hannan, Suprosanna Shit +4
Recent transformer-based offline video instance segmentation (VIS) approaches achieve encouraging results and significantly outperform online approaches. However, their reliance on…
Flamingo: a Visual Language Model for Few-Shot Learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc +24
Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Fl…
Relationformer: A Unified Framework for Image-to-Graph Generation
Suprosanna Shit, Rajat Koner, Bastian Wittmann +8
A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blo…
The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding
Volker Tresp, Sahand Sharifzadeh, Hang Li +2
We present a unified computational theory of an agent's perception and memory. In our model, perception, episodic memory, and semantic memory are realized by different operational…
Improving Scene Graph Classification by Exploiting Knowledge from Texts
Sahand Sharifzadeh, Sina Moayed Baharlou, Martin Schmitt +2
Training scene graph classification models requires a large amount of annotated image data. Meanwhile, scene graphs represent relational knowledge that can be modeled with symbolic…
Classification by Attention: Scene Graph Classification with Prior Knowledge
Sahand Sharifzadeh, Sina Moayed Baharlou, Volker Tresp
A major challenge in scene graph classification is that the appearance of objects and relations can be significantly different from one image to another. Previous works have addres…
PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings
Mehdi Ali, Max Berrendorf, Charles Tapley Hoyt +4
Recently, knowledge graph embeddings (KGEs) received significant attention, and several software libraries have been developed for training and evaluating KGEs. While each of them…
Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework
Mehdi Ali, Max Berrendorf, Charles Tapley Hoyt +6
The heterogeneity in recently published knowledge graph embedding models' implementations, training, and evaluation has made fair and thorough comparisons difficult. In order to as…
The Tensor Brain: Semantic Decoding for Perception and Memory
Volker Tresp, Sahand Sharifzadeh, Dario Konopatzki +1
We analyse perception and memory, using mathematical models for knowledge graphs and tensors, to gain insights into the corresponding functionalities of the human mind. Our discuss…
Improving Visual Relation Detection using Depth Maps
Sahand Sharifzadeh, Sina Moayed Baharlou, Max Berrendorf +2
Visual relation detection methods rely on object information extracted from RGB images such as 2D bounding boxes, feature maps, and predicted class probabilities. We argue that dep…
Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks
Sahand Sharifzadeh, Ioannis Chiotellis, Rudolph Triebel +1
We propose an inverse reinforcement learning (IRL) approach using Deep Q-Networks to extract the rewards in problems with large state spaces. We evaluate the performance of this ap…