Publications (38)
Applying deep learning techniques on medical corpora from the World Wide Web: a prototypical system and evaluation
Jose Antonio Miñarro-Giménez, Oscar MarÃn-Alonso, Matthias Samwald
BACKGROUND: The amount of biomedical literature is rapidly growing and it is becoming increasingly difficult to keep manually curated knowledge bases and ontologies up-to-date. In…
Global and local evaluation of link prediction tasks with neural embeddings
Asan Agibetov, Matthias Samwald
We focus our attention on the link prediction problem for knowledge graphs, which is treated herein as a binary classification task on neural embeddings of the entities. By compari…
ThoughtSource: A central hub for large language model reasoning data
Simon Ott, Konstantin Hebenstreit, Valentin Liévin +6
Large language models (LLMs) such as GPT-4 have recently demonstrated impressive results across a wide range of tasks. LLMs are still limited, however, in that they frequently fail…
A global analysis of metrics used for measuring performance in natural language processing
Kathrin Blagec, Georg Dorffner, Milad Moradi +2
Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and s…
TRIAGE: Ethical Benchmarking of AI Models Through Mass Casualty Simulations
Nathalie Maria Kirch, Konstantin Hebenstreit, Matthias Samwald
We present the TRIAGE Benchmark, a novel machine ethics (ME) benchmark that tests LLMs' ability to make ethical decisions during mass casualty incidents. It uses real-world ethical…
BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Jason Alan Fries, Leon Weber, Natasha Seelam +40
Training and evaluating language models increasingly requires the construction of meta-datasets --diverse collections of curated data with clear provenance. Natural language prompt…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
Kathrin Blagec, Jakob Kraiger, Wolfgang Frühwirt +1
Publicly accessible benchmarks that allow for assessing and comparing model performances are important drivers of progress in artificial intelligence (AI). While recent advances in…
Improving the robustness and accuracy of biomedical language models through adversarial training
Milad Moradi, Matthias Samwald
Deep transformer neural network models have improved the predictive accuracy of intelligent text processing systems in the biomedical domain. They have obtained state-of-the-art pe…
Explaining black-box text classifiers for disease-treatment information extraction
Milad Moradi, Matthias Samwald
Deep neural networks and other intricate Artificial Intelligence (AI) models have reached high levels of accuracy on many biomedical natural language processing tasks. However, the…
SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models
Simon Ott, Christian Meilicke, Matthias Samwald
Neural embedding-based machine learning models have shown promise for predicting novel links in knowledge graphs. Unfortunately, their practical utility is diminished by their lack…
What are the limits to biomedical research acceleration through general-purpose AI?
Konstantin Hebenstreit, Constantin Convalexius, Stephan Reichl +3
Although general-purpose artificial intelligence (GPAI) is widely expected to accelerate scientific discovery, its practical limits in biomedicine remain unclear. We assess this po…
OpenBioLink: A benchmarking framework for large-scale biomedical link prediction
Anna Breit, Simon Ott, Asan Agibetov +1
SUMMARY: Recently, novel machine-learning algorithms have shown potential for predicting undiscovered links in biomedical knowledge networks. However, dedicated benchmarks for meas…
Deep Learning, Natural Language Processing, and Explainable Artificial Intelligence in the Biomedical Domain
Milad Moradi, Matthias Samwald
In this article, we first give an introduction to artificial intelligence and its applications in biology and medicine in Section 1. Deep learning methods are then described in Sec…
An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
Konstantin Hebenstreit, Robert Praas, Louis P Kiesewetter +1
Emergent chain-of-thought (CoT) reasoning capabilities promise to improve performance and explainability of large language models (LLMs). However, uncertainties remain about how re…
A critical analysis of metrics used for measuring progress in artificial intelligence
Kathrin Blagec, Georg Dorffner, Milad Moradi +1
Comparing model performances on benchmark datasets is an integral part of measuring and driving progress in artificial intelligence. A model's performance on a benchmark dataset is…
Post-hoc explanation of black-box classifiers using confident itemsets
Milad Moradi, Matthias Samwald
Black-box Artificial Intelligence (AI) methods, e.g. deep neural networks, have been widely utilized to build predictive models that can extract complex relationships in a dataset…
Neural sentence embedding models for semantic similarity estimation in the biomedical domain
Kathrin Blagec, Hong Xu, Asan Agibetov +1
BACKGROUND: In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical…
Fast and scalable learning of neuro-symbolic representations of biomedical knowledge
Asan Agibetov, Matthias Samwald
In this work we address the problem of fast and scalable learning of neuro-symbolic representations for general biological knowledge. Based on a recently published comprehensive bi…
Towards an interoperable information infrastructure providing decision support for genomic medicine
Matthias Samwald, Holger Stenzhorn, Michel Dumontier +3
Genetic dispositions play a major role in individual disease risk and treatment response. Genomic medicine, in which medical decisions are refined by genetic information of particu…
Dividing the Ontology Alignment Task with Semantic Embeddings and Logic-based Modules
Ernesto Jiménez-Ruiz, Asan Agibetov, Jiaoyan Chen +2
Large ontologies still pose serious challenges to state-of-the-art ontology alignment systems. In this paper we present an approach that combines a neural embedding model and logic…
A critical review of methods and challenges in large language models
Milad Moradi, Ke Yan, David Colwell +2
This critical review provides an in-depth analysis of Large Language Models (LLMs), encompassing their foundational principles, diverse applications, and advanced training methodol…
Model-agnostic explainable artificial intelligence for object detection in image data
Milad Moradi, Ke Yan, David Colwell +2
In recent years, deep neural networks have been widely used for building high-performance Artificial Intelligence (AI) systems for computer vision applications. Object detection is…
Applying unsupervised keyphrase methods on concepts extracted from discharge sheets
Hoda Memarzadeh, Nasser Ghadiri, Matthias Samwald +1
Clinical notes containing valuable patient information are written by different health care providers with various scientific levels and writing styles. It might be helpful for cli…
A Study into patient similarity through representation learning from medical records
Hoda Memarzadeh, Nasser Ghadiri, Matthias Samwald +1
Patient similarity assessment, which identifies patients similar to a given patient, can help improve medical care. The assessment can be performed using Electronic Medical Records…
Evaluating the Robustness of Neural Language Models to Input Perturbations
Milad Moradi, Matthias Samwald
High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks. However, results for common benchmark dat…
CSMeD: Bridging the Dataset Gap in Automated Citation Screening for Systematic Literature Reviews
Wojciech Kusa, Oscar E. Mendoza, Matthias Samwald +2
Systematic literature reviews (SLRs) play an essential role in summarising, synthesising and validating scientific evidence. In recent years, there has been a growing interest in u…
Explaining Black-box Models for Biomedical Text Classification
Milad Moradi, Matthias Samwald
In this paper, we propose a novel method named Biomedical Confident Itemsets Explanation (BioCIE), aiming at post-hoc explanation of black-box machine learning models for biomedica…
Scalable and interpretable rule-based link prediction for large heterogeneous knowledge graphs
Simon Ott, Laura Graf, Asan Agibetov +2
Neural embedding-based machine learning models have shown promise for predicting novel links in biomedical knowledge graphs. Unfortunately, their practical utility is diminished by…
Breaking-down the Ontology Alignment Task with a Lexical Index and Neural Embeddings
Ernesto Jimenez-Ruiz, Asan Agibetov, Matthias Samwald +1
Large ontologies still pose serious challenges to state-of-the-art ontology alignment systems. In the paper we present an approach that combines a lexical index, a neural embedding…
GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
Milad Moradi, Kathrin Blagec, Florian Haberl +1
Deep neural language models have set new breakthroughs in many tasks of Natural Language Processing (NLP). Recent work has shown that deep transformer language models (pretrained o…
A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks
Kathrin Blagec, Adriano Barbosa-Silva, Simon Ott +1
Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track…
Clustering of Deep Contextualized Representations for Summarization of Biomedical Texts
Milad Moradi, Matthias Samwald
In recent years, summarizers that incorporate domain knowledge into the process of text summarization have outperformed generic methods, especially for summarization of biomedical…
Deep learning models are not robust against noise in clinical text
Milad Moradi, Kathrin Blagec, Matthias Samwald
Artificial Intelligence (AI) systems are attracting increasing interest in the medical domain due to their ability to learn complicated tasks that require human intelligence and ex…
Benchmarking neural embeddings for link prediction in knowledge graphs under semantic and structural changes
Asan Agibetov, Matthias Samwald
Recently, link prediction algorithms based on neural embeddings have gained tremendous popularity in the Semantic Web community, and are extensively used for knowledge graph comple…
A collection of principles for guiding and evaluating large language models
Konstantin Hebenstreit, Robert Praas, Matthias Samwald
Large language models (LLMs) demonstrate outstanding capabilities, but challenges remain regarding their ability to solve complex reasoning tasks, as well as their transparency, ro…
Multi-modal user interface control detection using cross-attention
Milad Moradi, Ke Yan, David Colwell +2
Detecting user interface (UI) controls from software screenshots is a critical task for automated testing, accessibility, and software analytics, yet it remains challenging due to…
Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec +2
Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI). However, recent studies raised concerns over the state of AI benchmarking, reporting issu…