134 citations · 459 across the 41 of their papers we have counts for
46 papers
Large language models surpass human experts in predicting neuroscience results
Xiaoliang Luo, Akilles Rechardt, Guangzhi Sun +36
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offe…
Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao, Yuanbin Qu, Konrad Staniszewski +5
Most language model pre-training frameworks concatenate multiple documents into fixed-length sequences and use causal masking to compute the likelihood of each token given its cont…
Using Natural Language Explanations to Improve Robustness of In-context Learning
Xuanli He, Yuxiang Wu, Oana-Maria Camburu +2
Recent studies demonstrated that large language models (LLMs) can excel in many tasks via in-context learning (ICL). However, recent works show that ICL-prompted models tend to pro…
Approximate Answering of Graph Queries
Michael Cochez, Dimitrios Alivanistos, Erik Arakelyan +6
Knowledge graphs (KGs) are inherently incomplete because of incomplete world knowledge and bias in what is the input to the KG. Additionally, world knowledge constantly expands and…
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models
Jean Kaddour, Oscar Key, Piotr Nawrot +2
The computation necessary for training Transformer-based language models has skyrocketed in recent years. This trend has motivated research on efficient training algorithms designe…
Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
Aryo Pradipta Gema, Pasquale Minervini, Luke Daines +2
Adapting pretrained language models to novel domains, such as clinical applications, traditionally involves retraining their entire set of parameters. Parameter-Efficient Fine-Tuni…
Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks
Aryo Pradipta Gema, Dominik Grabarczyk, Wolf De Wulf +5
Knowledge graphs are powerful tools for representing and organising complex biomedical data. Several knowledge graph embedding algorithms have been proposed to learn from and compl…
SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations
Jesus Solano, Mardhiyah Sanni, Oana-Maria Camburu +1
Models that generate natural language explanations (NLEs) for their predictions have recently gained increasing interest. However, this approach usually demands large datasets of h…
Atomic Inference for NLI with Generated Facts as Atoms
Joe Stacey, Pasquale Minervini, Haim Dubossarsky +2
With recent advances, neural models can achieve human-level performance on various natural language tasks. However, there are no guarantees that any explanations from these models…
Adapting Neural Link Predictors for Data-Efficient Complex Query Answering
Erik Arakelyan, Pasquale Minervini, Daniel Daza +2
Answering complex queries on incomplete knowledge graphs is a challenging task where a model needs to answer complex logical queries in the presence of missing knowledge. Prior wor…
Machine Learning-Assisted Recurrence Prediction for Early-Stage Non-Small-Cell Lung Cancer Patients
Adrianna Janik, Maria Torrente, Luca Costabello +14
Background: Stratifying cancer patients according to risk of relapse can personalize their care. In this work, we provide an answer to the following research question: How to utili…
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks
Yuxiang Wu, Yu Zhao, Baotian Hu +3
Access to external knowledge is essential for many natural language processing tasks, such as question answering and dialogue. Existing methods often rely on a parametric model tha…
Learning Discrete Directed Acyclic Graphs via Backpropagation
Andrew J. Wren, Pasquale Minervini, Luca Franceschi +1
Recently continuous relaxations have been proposed in order to learn Directed Acyclic Graphs (DAGs) from data by backpropagation, instead of using combinatorial optimization. Howev…
Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models
Pasquale Minervini, Luca Franceschi, Mathias Niepert
The integration of discrete algorithmic components in deep learning architectures has numerous applications. Recently, Implicit Maximum Likelihood Estimation (IMLE, Niepert, Minerv…
ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective
Yihong Chen, Pushkar Mishra, Luca Franceschi +3
Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs). Howe…
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
Joe Stacey, Pasquale Minervini, Haim Dubossarsky +1
Current Natural Language Inference (NLI) models achieve impressive results, sometimes outperforming humans when evaluating on in-distribution test sets. However, as these models ar…
XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking
Han Zhou, Ignacio Iacobacci, Pasquale Minervini
Dialogue State Tracking (DST), a crucial component of task-oriented dialogue (ToD) systems, keeps track of all important information pertaining to dialogue history: filling slots w…
MedDistant19: Towards an Accurate Benchmark for Broad-Coverage Biomedical Relation Extraction
Saadullah Amin, Pasquale Minervini, David Chang +2
Relation extraction in the biomedical domain is challenging due to the lack of labeled data and high annotation costs, needing domain experts. Distant supervision is commonly used…
Differentiable Reasoning over Long Stories -- Assessing Systematic Generalisation in Neural Models
Wanshui Li, Pasquale Minervini
Contemporary neural networks have achieved a series of developments and successes in many aspects; however, when exposed to data outside the training distribution, they may fail to…
A Probabilistic Framework for Knowledge Graph Data Augmentation
Jatin Chauhan, Priyanshu Gupta, Pasquale Minervini
We present NNMFAug, a probabilistic framework to perform data augmentation for the task of knowledge graph completion to counter the problem of data scarcity, which can enhance the…
Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations
Yihong Chen, Pasquale Minervini, Sebastian Riedel +1
Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for mul…
Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints
Yuxiang Wu, Pasquale Minervini, Pontus Stenetorp +1
Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems. However, current AC approaches require tuning…
Combining Inductive and Deductive Reasoning for Query Answering over Incomplete Knowledge Graphs
Medina Andresel, Trung-Kien Tran, Csaba Domokos +2
Current methods for embedding-based query answering over incomplete Knowledge Graphs (KGs) only focus on inductive reasoning, i.e., predicting answers by learning patterns from the…
Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions
Mathias Niepert, Pasquale Minervini, Luca Franceschi
Combining discrete probability distributions and combinatorial optimization problems with neural network components has numerous applications but poses several challenges. We propo…