Learnt dynamics generalizes across tasks, datasets, and populations
arXiv:1912.03130
Abstract
Differentiating multivariate dynamic signals is a difficult learning problem as the feature space may be large yet often only a few training examples are available. Traditional approaches to this problem either proceed from handcrafted features or require large datasets to combat the m >> n problem. In this paper, we show that the source of the problem---signal dynamics---can be used to our advantage and noticeably improve classification performance on a range of discrimination tasks when training data is scarce. We demonstrate that self-supervised pre-training guided by signal dynamics produces embedding that generalizes across tasks, datasets, data collection sites, and data distributions. We perform an extensive evaluation of this approach on a range of tasks including simulated data, keyword detection problem, and a range of functional neuroimaging data, where we show that a single embedding learnt on healthy subjects generalizes across a number of disorders, age groups, and datasets.
arXiv comment11 pages, 12 figures. arXiv admin note: text overlap with arXiv:1911.06813
References in corpus (4)
- Distributed Representations of Words and Phrases and their Compositionality
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- On Mutual Information Maximization for Representation Learning
- Improving Classification Rate of Schizophrenia Using a Multimodal Multi-Layer Perceptron Model with Structural and Functional MR