A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
arXiv:2007.10185
Abstract
Multi-task learning (MTL) is a machine learning technique aiming to improve model performance by leveraging information across many tasks. It has been used extensively on various data modalities, including electronic health record (EHR) data. However, despite significant use on EHR data, there has been little systematic investigation of the utility of MTL across the diverse set of possible tasks and training schemes of interest in healthcare. In this work, we examine MTL across a battery of tasks on EHR time-series data. We find that while MTL does suffer from common negative transfer, we can realize significant gains via MTL pre-training combined with single-task fine-tuning. We demonstrate that these gains can be achieved in a task-independent manner and offer not only minor improvements under traditional learning, but also notable gains in a few-shot learning context, thereby suggesting this could be a scalable vehicle to offer improved performance in important healthcare contexts.
References in corpus (8)
- An Overview of Multi-Task Learning in Deep Neural Networks
- A Survey on Multi-Task Learning
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
- A Framework for Individualizing Predictions of Disease Trajectories by Exploiting Multi-Resolution Structure
- Understanding and Improving Information Transfer in Multi-Task Learning
- Multi-task Learning via Adaptation to Similar Tasks for Mortality Prediction of Diverse Rare Diseases
- Multi-Task Learning with Incomplete Data for Healthcare