CADET: Debugging and Fixing Misconfigurations using Counterfactual Reasoning

Abstract

Modern computing platforms are highly-configurable with thousands of interacting configurations. However, configuring these systems is challenging. Erroneous configurations can cause unexpected non-functional faults. This paper proposes CADET (short for Causal Debugging Toolkit) that enables users to identify, explain, and fix the root cause of non-functional faults early and in a principled fashion. CADET builds a causal model by observing the performance of the system under different configurations. Then, it uses casual path extraction followed by counterfactual reasoning over the causal model to: (a) identify the root causes of non-functional faults, (b) estimate the effects of various configurable parameters on the performance objective(s), and (c) prescribe candidate repairs to the relevant configuration options to fix the non-functional fault. We evaluated CADET on 5 highly-configurable systems deployed on 3 NVIDIA Jetson systems-on-chip. We compare CADET with state-of-the-art configuration optimization and ML-based debugging approaches. The experimental results indicate that CADET can find effective repairs for faults in multiple non-functional properties with (at most) 17% more accuracy, 28% higher gain, and $40\times$ speed-up than other ML-based performance debugging methods. Compared to multi-objective optimization approaches, CADET can find fixes (at most) $9\times$ faster with comparable or better performance gain. Our case study of non-functional faults reported in NVIDIA's forum show that CADET can find $14$ better repairs than the experts' advice in less than 30 minutes.

References in corpus (10)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Predictive Entropy Search for Multi-objective Bayesian Optimization
• Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack
• Markovian acyclic directed mixed graphs for discrete data
• Towards Automated Performance Bug Identification in Python
• Less is More: Minimizing Code Reorganization using XTREE
• Applications of Common Entropy for Causal Inference
• Predicting Performance of Software Configurations: There is no Silver Bullet
• Transfer Learning for Performance Modeling of Deep Neural Network Systems
• Transfer Learning for Performance Modeling of Configurable Systems: A Causal Analysis

Cited by in corpus (2)

• Causality in Configurable Software Systems
• Automated Misconfiguration Repair of Configurable Cyber-Physical Systems with Search: an Industrial Case Study on Elevator Dispatching Algorithms