Toward a Theory of Causation for Interpreting Neural Code Models

Abstract

Neural Language Models of Code, or Neural Code Models (NCMs), are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models arrive at decisions. To this end, this paper introduces $d{o}_{code}$, a post hoc interpretability method specific to NCMs that is capable of explaining model predictions. $d{o}_{code}$ is based upon causal inference to enable programming language-oriented explanations. While the theoretical underpinnings of $d{o}_{code}$ are extensible to exploring different model properties, we provide a concrete instantiation that aims to mitigate the impact of spurious correlations by grounding explanations of model behavior in properties of programming languages. To demonstrate the practical benefit of $d{o}_{code}$, we illustrate the insights that our framework can provide by performing a case study on two popular deep learning architectures and ten NCMs. The results of this case study illustrate that our studied NCMs are sensitive to changes in code syntax. All our NCMs, except for the BERT-like model, statistically learn to predict tokens related to blocks of code (\eg brackets, parenthesis, semicolon) with less confounding bias as compared to other programming language constructs. These insights demonstrate the potential of $d{o}_{code}$ as a useful method to detect and facilitate the elimination of confounding bias in NCMs.

Accepted to appear in IEEE Transactions on Software Engineering

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Towards A Rigorous Science of Interpretable Machine Learning
• Evaluating Large Language Models Trained on Code
• Visualizing and Understanding Recurrent Networks
• CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
• SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
• Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities
• On Learning Meaningful Assert Statements for Unit Test Cases
• To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation
• Maybe Deep Neural Networks are the Best Choice for Modeling Source Code
• Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations
• DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions
• Deep Transfer Learning for Source Code Modeling
• Explainable AI for Pre-Trained Code Models: What Do They Learn? When They Do Not Work?
• On the Relationship Between Explanation and Prediction: A Causal View