On a Catalogue of Metrics for Evaluating Commercial Cloud Services
arXiv:1302.1954 · doi:10.1109/Grid.2012.15
Abstract
Given the continually increasing amount of commercial Cloud services in the market, evaluation of different services plays a significant role in cost-benefit analysis or decision making for choosing Cloud Computing. In particular, employing suitable metrics is essential in evaluation implementations. However, to the best of our knowledge, there is not any systematic discussion about metrics for evaluating Cloud services. By using the method of Systematic Literature Review (SLR), we have collected the de facto metrics adopted in the existing Cloud services evaluation work. The collected metrics were arranged following different Cloud service features to be evaluated, which essentially constructed an evaluation metrics catalogue, as shown in this paper. This metrics catalogue can be used to facilitate the future practice and research in the area of Cloud services evaluation. Moreover, considering metrics selection is a prerequisite of benchmark selection in evaluation implementations, this work also supplements the existing research in benchmarking the commercial Cloud services.
arXiv comment10 pages, Proceedings of the 13th ACM/IEEE International Conference on Grid Computing (Grid 2012), pp. 164-173, Beijing, China, September 20-23, 2012
References in corpus (2)
Cited by in corpus (16)
- Function-as-a-Service Performance Evaluation: A Multivocal Literature Review
- On Evaluating Commercial Cloud Services: A Systematic Review
- Cloud Benchmarking for Performance
- Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics
- Container-Based Cloud Virtual Machine Benchmarking
- Early Observations on Performance of Google Compute Engine for Scientific Computing
- A Factor Framework for Experimental Design for Performance Evaluation of Commercial Cloud Services
- DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool
- Cloud Benchmarking For Maximising Performance of Scientific Applications
- Boosting Metrics for Cloud Services Evaluation -- The Last Mile of Using Benchmark Suites
- DoKnowMe: Towards a Domain Knowledge-driven Methodology for Performance Evaluation
- DeFog: Fog Computing Benchmarks
- Patterns in the Chaos - a Study of Performance Variation and Predictability in Public IaaS Clouds
- A note on "The Need for End-to-End Evaluation of Cloud Availability"
- Determinants of a Successful Migration to Cloud Computing in Iranian Telecommunication Industry
- Building an Expert System for Evaluation of Commercial Cloud Services