NSML: A Machine Learning Platform That Enables You to Focus on Your Models

Abstract

Machine learning libraries such as TensorFlow and PyTorch simplify model implementation. However, researchers are still required to perform a non-trivial amount of manual tasks such as GPU allocation, training status tracking, and comparison of models with different hyperparameter settings. We propose a system to handle these tasks and help researchers focus on models. We present the requirements of the system based on a collection of discussions from an online study group comprising 25k members. These include automatic GPU allocation, learning status visualization, handling model parameter snapshots as well as hyperparameter modification during learning, and comparison of performance metrics between models via a leaderboard. We describe the system architecture that fulfills these requirements and present a proof-of-concept implementation, NAVER Smart Machine Learning (NSML). We test the system and confirm substantial efficiency improvements for model development.

8 pages, 4figures

References in corpus (9)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Convolutional Sequence to Sequence Learning
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Densely Connected Convolutional Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• A Structured Self-attentive Sentence Embedding
• YouTube-8M: A Large-Scale Video Classification Benchmark
• YOLO9000: Better, Faster, Stronger
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech

Cited by in corpus (34)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• In defence of metric learning for speaker recognition
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
• Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs
• An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
• Toward Interpretable Music Tagging with Self-Attention
• HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information
• Tripartite Heterogeneous Graph Propagation for Large-scale Social Recommendation
• Efficient Dialogue State Tracking by Selectively Overwriting Memory
• Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
• CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
• Multi-Manifold Learning for Large-scale Targeted Advertising System
• ClovaCall: Korean Goal-Oriented Dialog Speech Corpus for Automatic Speech Recognition of Contact Centers
• Which Strategies Matter for Noisy Label Classification? Insight into Loss and Uncertainty
• HPCClusterScape: Increasing Transparency and Efficiency of Shared High-Performance Computing Clusters for Large-scale AI Models
• Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning
• Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
• Orchestrate: Infrastructure for Enabling Parallelism during Hyperparameter Optimization
• M2FN: Multi-step Modality Fusion for Advertisement Image Assessment
• Scaling Law for Recommendation Models: Towards General-purpose User Representations
• Hop Sampling: A Simple Regularized Graph Learning for Non-Stationary Environments
• Which Ads to Show? Advertisement Image Assessment with Auxiliary Information via Multi-step Modality Fusion
• Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
• Graphs, Entities, and Step Mixture
• Large Product Key Memory for Pretrained Language Models
• Neural Approximation of an Auto-Regressive Process through Confidence Guided Sampling
• Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering
• NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation
• Subword Language Model for Query Auto-Completion