Real-World Image Datasets for Federated Learning

Abstract

Federated learning is a new machine learning paradigm which allows data parties to build machine learning models collaboratively while keeping their data secure and private. While research efforts on federated learning have been growing tremendously in the past two years, most existing works still depend on pre-existing public datasets and artificial partitions to simulate data federations due to the lack of high-quality labeled data generated from real-world edge applications. Consequently, advances on benchmark and model evaluations for federated learning have been lagging behind. In this paper, we introduce a real-world image dataset. The dataset contains more than 900 images generated from 26 street cameras and 7 object categories annotated with detailed bounding box. The data distribution is non-IID and unbalanced, reflecting the characteristic real-world federated learning scenarios. Based on this dataset, we implemented two mainstream object detection algorithms (YOLO and Faster R-CNN) and provided an extensive benchmark on model performance, efficiency, and communication in a federated learning setting. Both the dataset and algorithms are made publicly available.

arXiv commentThis paper is published at the 2nd International Workshop on Federated Learning for Data Privacy and Confidentiality, in Conjunction with NeurIPS 2019 (FL-NeurIPS 19)

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• YOLOv3: An Incremental Improvement
• Federated Learning for Mobile Keyboard Prediction
• Towards Federated Learning at Scale: System Design
• Learning Differentially Private Recurrent Language Models
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• A generic framework for privacy preserving deep learning
• LEAF: A Benchmark for Federated Settings
• Simultaneous Detection and Segmentation
• SecureBoost: A Lossless Federated Learning Framework
• Federated Learning Of Out-Of-Vocabulary Words
• Federated Learning for Keyword Spotting

Cited by in corpus (13)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• The Non-IID Data Quagmire of Decentralized Machine Learning
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• Auto-weighted Robust Federated Learning with Corrupted Data Sources
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• FedEval: A Holistic Evaluation Framework for Federated Learning
• Federated Learning for Commercial Image Sources
• Towards Unsupervised Domain Adaptation for Deep Face Recognition under Privacy Constraints via Federated Learning
• Federated Learning on Non-IID Data: A Survey
• FedOCR: Communication-Efficient Federated Learning for Scene Text Recognition
• An improved helmet detection method for YOLOv3 on an unbalanced dataset
• FLBench: A Benchmark Suite for Federated Learning