78 citations · 224 across the 8 of their papers we have counts for
10 papers
VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval
Simon Ott, Yufan Chen, Ruiping Liu +6
Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap betwee…
ThoughtSource: A central hub for large language model reasoning data
Simon Ott, Konstantin Hebenstreit, Valentin Liévin +6
Large language models (LLMs) such as GPT-4 have recently demonstrated impressive results across a wide range of tasks. LLMs are still limited, however, in that they frequently fail…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Jason Alan Fries, Leon Weber, Natasha Seelam +40
Training and evaluating language models increasingly requires the construction of meta-datasets --diverse collections of curated data with clear provenance. Natural language prompt…
A global analysis of metrics used for measuring performance in natural language processing
Kathrin Blagec, Georg Dorffner, Milad Moradi +2
Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and s…
Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec +2
Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI). However, recent studies raised concerns over the state of AI benchmarking, reporting issu…
A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks
Kathrin Blagec, Adriano Barbosa-Silva, Simon Ott +1
Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track…
SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models
Simon Ott, Christian Meilicke, Matthias Samwald
Neural embedding-based machine learning models have shown promise for predicting novel links in knowledge graphs. Unfortunately, their practical utility is diminished by their lack…
Scalable and interpretable rule-based link prediction for large heterogeneous knowledge graphs
Simon Ott, Laura Graf, Asan Agibetov +2
Neural embedding-based machine learning models have shown promise for predicting novel links in biomedical knowledge graphs. Unfortunately, their practical utility is diminished by…
OpenBioLink: A benchmarking framework for large-scale biomedical link prediction
Anna Breit, Simon Ott, Asan Agibetov +1
SUMMARY: Recently, novel machine-learning algorithms have shown potential for predicting undiscovered links in biomedical knowledge networks. However, dedicated benchmarks for meas…