Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources

Abstract

Machine learning (ML) in healthcare presents numerous opportunities for enhancing patient care, population health, and healthcare providers' workflows. However, the real-world clinical and cost benefits remain limited due to challenges in data privacy, heterogeneous data sources, and the inability to fully leverage multiple data modalities. In this perspective paper, we introduce "patchwork learning" (PL), a novel paradigm that addresses these limitations by integrating information from disparate datasets composed of different data modalities (e.g., clinical free-text, medical images, omics) and distributed across separate and secure sites. PL allows the simultaneous utilization of complementary data sources while preserving data privacy, enabling the development of more holistic and generalizable ML models. We present the concept of patchwork learning and its current implementations in healthcare, exploring the potential opportunities and applicable data sources for addressing various healthcare challenges. PL leverages bridging modalities or overlapping feature spaces across sites to facilitate information sharing and impute missing data, thereby addressing related prediction tasks. We discuss the challenges associated with PL, many of which are shared by federated and multimodal learning, and provide recommendations for future research in this field. By offering a more comprehensive approach to healthcare data integration, patchwork learning has the potential to revolutionize the clinical applicability of ML models. This paradigm promises to strike a balance between personalization and generalizability, ultimately enhancing patient experiences, improving population health, and optimizing healthcare providers' workflows.

References in corpus (32)

• Denoising Diffusion Probabilistic Models
• The Future of Digital Health with Federated Learning
• Towards Personalized Federated Learning
• Domain Generalization: A Survey
• Learning under Concept Drift: A Review
• Domain Generalization via Invariant Feature Representation
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Integrated multimodal artificial intelligence framework for healthcare applications
• PaLM-E: An Embodied Multimodal Language Model
• Fair Resource Allocation in Federated Learning
• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
• Agnostic Federated Learning
• Learning Factorized Multimodal Representations
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• One-Shot Federated Learning
• Joint Multimodal Learning with Deep Generative Models
• Distilled One-Shot Federated Learning
• Federated Multi-view Matrix Factorization for Personalized Recommendations
• Federated Learning for Healthcare Informatics
• A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
• FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings
• FedNorm: Modality-Based Normalization in Federated Learning for Multi-Modal Liver Segmentation
• A Federated Multi-View Deep Learning Framework for Privacy-Preserving Recommendations
• Incentivizing Federated Learning
• A Multimodal Sensor Fusion Framework Robust to Missing Modalities for Person Recognition
• GenoML: Automated Machine Learning for Genomics
• An Adaptive Kernel Approach to Federated Learning of Heterogeneous Causal Effects
• FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare
• On the Power of Foundation Models
• Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxies
• Image Generation with Multimodal Priors using Denoising Diffusion Probabilistic Models
• Federated Transfer Learning with Multimodal Data