Diversity in Machine Learning

Abstract

Machine learning methods have achieved good performance and been widely applied in various real-world applications. They can learn the model adaptively and be better fit for special requirements of different tasks. Generally, a good machine learning system is composed of plentiful training data, a good model training process, and an accurate inference. Many factors can affect the performance of the machine learning process, among which the diversity of the machine learning process is an important one. The diversity can help each procedure to guarantee a total good machine learning: diversity of the training data ensures that the training data can provide more discriminative information for the model, diversity of the learned model (diversity in parameters of each model or diversity among different base models) makes each parameter/model capture unique or complement information and the diversity in inference can provide multiple choices each of which corresponds to a specific plausible local optimal result. Even though the diversity plays an important role in machine learning process, there is no systematical analysis of the diversification in machine learning system. In this paper, we systematically summarize the methods to make data diversification, model diversification, and inference diversification in the machine learning process, respectively. In addition, the typical applications where the diversity technology improved the machine learning performance have been surveyed, including the remote sensing imaging tasks, machine translation, camera relocalization, image segmentation, object detection, topic modeling, and others. Finally, we discuss some challenges of the diversity technology in machine learning and point out some directions in future work.

arXiv commentAccepted by IEEE Access

References in corpus (21)

• Determinantal point processes for machine learning
• Land Use Classification in Remote Sensing Images by Convolutional Neural Networks
• Determinantal point process models and statistical inference : Extended version
• A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields
• Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks
• Mutual Information and Diverse Decoding Improve Neural Machine Translation
• Thermodynamics of natural images
• Learning the Parameters of Determinantal Point Process Kernels
• Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
• Fast determinantal point processes via distortion-free intermediate sampling
• Diversified Hidden Markov Models for Sequential Labeling
• On the Generalization Error Bounds of Neural Networks under Diversity-Inducing Mutual Angular Regularization
• Maximum Entropy Discrimination Markov Networks
• Fixed-point algorithms for learning determinantal point processes
• Learning to Diversify via Weighted Kernels for Classifier Ensemble
• An ensemble diversity approach to supervised binary hashing
• Diversity-Promoting Bayesian Learning of Latent Variable Models
• Inference for determinantal point processes without spectral knowledge
• Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization
• Maximizing Diversity for Multimodal Optimization
• An End-to-End Joint Unsupervised Learning of Deep Model and Pseudo-Classes for Remote Sensing Scene Representation

Cited by in corpus (24)

• Black-Box Testing of Deep Neural Networks Through Test Case Diversity
• LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing
• Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries
• Coverage Guided Testing for Recurrent Neural Networks
• DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks
• Finding Quasars behind the Galactic Plane. I. Candidate Selections with Transfer Learning
• Smartphone Sensing for the Well-being of Young Adults: A Review
• On Misbehaviour and Fault Tolerance in Machine Learning Systems
• Crowdsourcing the Perception of Machine Teaching
• The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations
• Complex Daily Activities, Country-Level Diversity, and Smartphone Sensing: A Study in Denmark, Italy, Mongolia, Paraguay, and UK
• Metarobotics for Industry and Society: Vision, Technologies, and Opportunities
• VulScribeR: Exploring RAG-based Vulnerability Augmentation with LLMs
• WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Users
• Learning About Social Context from Smartphone Data: Generalization Across Countries and Daily Life Moments
• Open Domain Generalization with Domain-Augmented Meta-Learning
• Spatial LibriSpeech: An Augmented Dataset for Spatial Audio Learning
• Experimental Standards for Deep Learning in Natural Language Processing Research
• LAQP: Learning-based Approximate Query Processing
• Improving Image Clustering through Sample Ranking and Its Application to remote--sensing images
• EIT Spectroscopy of Rydberg Levels Dressed by Linearly Polarized RF fields: Complementary Angular Response for Two Types of Transition Ladders
• Deep Manifold Embedding for Hyperspectral Image Classification
• XPROAX-Local explanations for text classification with progressive neighborhood approximation
• Understanding Uncertainty of Edge Computing: New Principle and Design Approach