Enabling On-Demand Database Computing with MIT SuperCloud Database Management System
arXiv:1506.08506 · doi:10.1109/HPEC.2015.7322482
Abstract
The MIT SuperCloud database management system allows for rapid creation and flexible execution of a variety of the latest scientific databases, including Apache Accumulo and SciDB. It is designed to permit these databases to run on a High Performance Computing Cluster (HPCC) platform as seamlessly as any other HPCC job. It ensures the seamless migration of the databases to the resources assigned by the HPCC scheduler and centralized storage of the database files when not running. It also permits snapshotting of databases to allow researchers to experiment and push the limits of the technology without concerns for data or productivity loss if the database becomes unstable.
6 pages; accepted to IEEE High Performance Extreme Computing (HPEC) conference 2015. arXiv admin note: text overlap with arXiv:1406.4923
References in corpus (1)
Cited by in corpus (20)
- Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
- Measuring the Impact of Spectre and Meltdown
- LLMapReduce: Multi-Level Map-Reduce for High Performance Data Analysis
- Scalability of VM Provisioning Systems
- MIT SuperCloud Portal Workspace: Enabling HPC Web Application Deployment
- Benchmarking Data Analysis and Machine Learning Applications on the Intel KNL Many-Core Processor
- Node-Based Job Scheduling for Large Scale Simulations of Short Running Jobs
- Enhancing HPC Security with a User-Based Firewall
- Maneuver Identification Challenge
- Lustre, Hadoop, Accumulo
- AI Enabling Technologies: A Survey
- Best of Both Worlds: High Performance Interactive and Batch Launching
- Lessons Learned from a Decade of Providing Interactive, On-Demand High Performance Computing to Scientists and Engineers
- 3D Real-Time Supercomputer Monitoring
- Database Operations in D4M.jl
- Securing HPC using Federated Authentication
- D4M 3.0: Extended Database and Language Capabilities
- Technical Report: Developing a Working Data Hub
- Scaling Big Data Platform for Big Data Pipeline
- HPC with Enhanced User Separation