Publications (39)
SemEval-2017 Task 3: Community Question Answering
Preslav Nakov, Doris Hoogeveen, LluÃs MÃ rquez +4
We describe SemEval-2017 Task 3 on Community Question Answering. This year, we reran the four subtasks from SemEval-2016:(A) Question-Comment Similarity,(B) Question-Question Simil…
Improving Text-based Early Prediction by Distillation from Privileged Time-Series Text
Jinghui Liu, Daniel Capurro, Anthony Nguyen +1
Modeling text-based time-series to make prediction about a future event or outcome is an important task with a wide range of applications. The standard approach is to train and tes…
WikiUMLS: Aligning UMLS to Wikipedia via Cross-lingual Neural Ranking
Afshin Rahimi, Timothy Baldwin, Karin Verspoor
We present our work on aligning the Unified Medical Language System (UMLS) to Wikipedia, to facilitate manual alignment of the two resources. We propose a cross-lingual neural rera…
LED down the rabbit hole: exploring the potential of global attention for biomedical multi-document summarisation
Yulia Otmakhova, Hung Thinh Truong, Timothy Baldwin +3
In this paper we report on our submission to the Multidocument Summarisation for Literature Review (MSLR) shared task. Specifically, we adapt PRIMERA (Xiao et al., 2022) to the bio…
Principles from Clinical Research for NLP Model Generalization
Aparna Elangovan, Jiayuan He, Yuan Li +1
The NLP community typically relies on performance of a model on a held-out test set to assess generalization. Performance drops observed in datasets outside of official test sets a…
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation
Mingjie Li, Wenjia Cai, Karin Verspoor +3
Automatic generation of ophthalmic reports using data-driven neural networks has great potential in clinical practice. When writing a report, ophthalmologists make inferences with…
Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings
Zenan Zhai, Dat Quoc Nguyen, Saber A. Akhondi +5
Chemical patents are an important resource for chemical information. However, few chemical Named Entity Recognition (NER) systems have been evaluated on patent documents, due in pa…
Adjusting for Chance Clustering Comparison Measures
Simone Romano, Nguyen Xuan Vinh, James Bailey +1
Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Ad…
Assigning function to protein-protein interactions: a weakly supervised BioBERT based approach using PubMed abstracts
Aparna Elangovan, Melissa Davis, Karin Verspoor
Motivation: Protein-protein interactions (PPI) are critical to the function of proteins in both normal and diseased cells, and many critical protein functions are mediated by inter…
Improving negation detection with negation-focused pre-training
Thinh Hung Truong, Timothy Baldwin, Trevor Cohn +1
Negation is a common linguistic feature that is crucial in many language understanding tasks, yet it remains a hard problem due to diversity in its expression in different types of…
Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation
Aparna Elangovan, Jiayuan He, Karin Verspoor
Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). However, the presence…
EMBRE: Entity-aware Masking for Biomedical Relation Extraction
Mingjie Li, Karin Verspoor
Information extraction techniques, including named entity recognition (NER) and relation extraction (RE), are crucial in many domains to support making sense of vast amounts of uns…
MPVNN: Mutated Pathway Visible Neural Network Architecture for Interpretable Prediction of Cancer-specific Survival Risk
Gourab Ghosh Roy, Nicholas Geard, Karin Verspoor +1
Survival risk prediction using gene expression data is important in making treatment decisions in cancer. Standard neural network (NN) survival analysis models are black boxes with…
Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE): Manifesto
Christine Bauer, Li Chen, Nicola Ferro +19
During the workshop, we deeply discussed what CONversational Information ACcess (CONIAC) is and its unique features, proposing a world model abstracting it, and defined the Convers…
Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition
Zenan Zhai, Dat Quoc Nguyen, Karin Verspoor
We compare the use of LSTM-based and CNN-based character-level word embeddings in BiLSTM-CRF models to approach chemical and disease named entity recognition (NER) tasks. Empirical…
Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings
Dat Quoc Nguyen, Karin Verspoor
We investigate the incorporation of character-based word representations into a standard CNN-based relation extraction model. We experiment with two common neural architectures, CN…
Collective Human Opinions in Semantic Textual Similarity
Yuxia Wang, Shimin Tao, Ning Xie +3
Despite the subjective nature of semantic textual similarity (STS) and pervasive disagreements in STS annotation, existing benchmarks have used averaged human ratings as the gold s…
Graph Transformers: A Survey
Ahsan Shehzad, Feng Xia, Shagufta Abid +4
Graph transformers are a recent advancement in machine learning, offering a new class of neural network models for graph-structured data. The synergy between transformers and graph…
Uncovering protein interaction in abstracts and text using a novel linear model and word proximity networks
Alaa Abi-Haidar, Jasleen Kaur, Ana G. Maguitman +5
We participated in three of the protein-protein interaction subtasks of the Second BioCreative Challenge: classification of abstracts relevant for protein-protein interaction (IAS)…
COVID-SEE: Scientific Evidence Explorer for COVID-19 Related Research
Karin Verspoor, Simon Å uster, Yulia Otmakhova +7
We present COVID-SEE, a system for medical literature discovery based on the concept of information exploration, which builds on several distinct text analysis and natural language…
Analysing health professionals' learning interactions in online social networks: A social network analysis approach
Xin Li, Kathleen Gray, Karin Verspoor +1
Online Social Networking may be a way to support health professionals' need for continuous learning through interaction with peers and experts. Understanding and evaluating such le…
A bag-of-concepts model improves relation extraction in a narrow knowledge domain with limited data
Jiyu Chen, Karin Verspoor, Zenan Zhai
This paper focuses on a traditional relation extraction task in the context of limited annotated data and a narrow knowledge domain. We explore this task with a clinical corpus con…
RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settings
Wei Han, David Martinez, Anna Khanina +2
A common strategy in transfer learning is few shot fine-tuning, but its success is highly dependent on the quality of samples selected as training examples. Active learning methods…
How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts
Minh-Vuong Nguyen, Fatemeh Shiri, Zhuang Li +1
Large Language Models (LLMs) are increasingly being explored for clinical question answering and decision support, yet safe deployment critically requires reliable handling of pati…
Deep Outdated Fact Detection in Knowledge Graphs
Huiling Tu, Shuo Yu, Vidya Saikrishna +2
Knowledge graphs (KGs) have garnered significant attention for their vast potential across diverse domains. However, the issue of outdated facts poses a challenge to KGs, affecting…
Large-scale protein-protein post-translational modification extraction with distant supervision and confidence calibrated BioBERT
Aparna Elangovan, Yuan Li, Douglas E. V. Pires +2
Protein-protein interactions (PPIs) are critical to normal cellular function and are related to many disease pathways. However, only 4% of PPIs are annotated with PTMs in biologica…
Impact of detecting clinical trial elements in exploration of COVID-19 literature
Simon Å uster, Karin Verspoor, Timothy Baldwin +4
The COVID-19 pandemic has driven ever-greater demand for tools which enable efficient exploration of biomedical literature. Although semi-structured information resulting from conc…
Revisiting subword tokenization: A case study on affixal negation in large language models
Thinh Hung Truong, Yulia Otmakhova, Karin Verspoor +2
In this work, we measure the impact of affixal negation on modern English large language models (LLMs). In affixal negation, the negated meaning is expressed through a negative mor…
ITTC @ TREC 2021 Clinical Trials Track
Thinh Hung Truong, Yulia Otmakhova, Rahmad Mahendra +6
This paper describes the submissions of the Natural Language Processing (NLP) team from the Australian Research Council Industrial Transformation Training Centre (ITTC) for Cogniti…
A Framework to Adjust Dependency Measure Estimates for Chance
Simone Romano, Nguyen Xuan Vinh, James Bailey +1
Estimating the strength of dependency between two variables is fundamental for exploratory analysis and many other applications in data mining. For example: non-linear dependencies…
Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation
Thinh Hung Truong, Yulia Otmakhova, Timothy Baldwin +3
Negation is poorly captured by current language models, although the extent of this problem is not widely understood. We introduce a natural language inference (NLI) test suite to…
Effects of Human Adversarial and Affable Samples on BERT Generalization
Aparna Elangovan, Jiayuan He, Yuan Li +1
BERT-based models have had strong performance on leaderboards, yet have been demonstrably worse in real-world settings requiring generalization. Limited quantities of training data…
End-to-end neural relation extraction using deep biaffine attention
Dat Quoc Nguyen, Karin Verspoor
We propose a neural network model for joint extraction of named entities and relations between them, without any hand-crafted features. The key contribution of our model is to exte…
A Framework for Transparent Reporting of Data Quality Analysis Across the Clinical Electronic Health Record Data Lifecycle
Melinda Wassell, Kerryn Butler-Henderson, Karin Verspoor
Data quality (DQ) and transparency of secondary data are critical factors that delay the adoption of clinical AI models and affect clinician trust in them. Many DQ studies fail to…
From POS tagging to dependency parsing for biomedical event extraction
Dat Quoc Nguyen, Karin Verspoor
Background: Given the importance of relation or event extraction from biomedical research publications to support knowledge capture and synthesis, and the strong dependency of appr…
The Illusion of AI Expertise Under Uncertainty: Navigating Elusive Ground Truth via a Probabilistic Paradigm
Aparna Elangovan, Lei Xu, Mahsa Elyasi +8
Benchmarking the capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth an…
Language models are not naysayers: An analysis of language models on negation benchmarks
Thinh Hung Truong, Timothy Baldwin, Karin Verspoor +1
Negation has been shown to be a major bottleneck for masked language models, such as BERT. However, whether this finding still holds for larger-sized auto-regressive language model…
Learning Robust Negation Text Representations
Thinh Hung Truong, Karin Verspoor, Trevor Cohn +1
Despite rapid adoption of autoregressive large language models, smaller text encoders still play an important role in text understanding tasks that require rich contextualized repr…
An improved neural network model for joint POS tagging and dependency parsing
Dat Quoc Nguyen, Karin Verspoor
We propose a novel neural network model for joint part-of-speech (POS) tagging and dependency parsing. Our model extends the well-known BIST graph-based dependency parser (Kiperwas…