Understanding Neural Code Intelligence Through Program Simplification

Abstract

A wide range of code intelligence (CI) tools, powered by deep neural networks, have been developed recently to improve programming productivity and perform program analysis. To reliably use such tools, developers often need to reason about the behavior of the underlying models and the factors that affect them. This is especially challenging for tools backed by deep neural networks. Various methods have tried to reduce this opacity in the vein of "transparent/interpretable-AI". However, these approaches are often specific to a particular set of network architectures, even requiring access to the network's parameters. This makes them difficult to use for the average programmer, which hinders the reliable adoption of neural CI systems. In this paper, we propose a simple, model-agnostic approach to identify critical input features for models in CI systems, by drawing on software debugging research, specifically delta debugging. Our approach, SIVAND, uses simplification techniques that reduce the size of input programs of a CI model while preserving the predictions of the model. We show that this approach yields remarkably small outputs and is broadly applicable across many model architectures and problem domains. We find that the models in our experiments often rely heavily on just a few syntactic features in input programs. We believe that SIVAND's extracted features may help understand neural CI systems' predictions and learned behavior.

arXiv commentThe 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'21)

References in corpus (10)

• Explaining and Harnessing Adversarial Examples
• Axiomatic Attribution for Deep Networks
• Interpretable machine learning: definitions, methods, and applications
• The Shattered Gradients Problem: If resnets are the answer, then what is the question?
• Semantic Robustness of Models of Source Code
• Neural Program Repair by Jointly Learning to Localize and Repair
• Adversarial Robustness for Code
• How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset
• Testing Neural Program Analyzers
• Evaluation of Generalizability of Neural Program Analyzers under Semantic-Preserving Transformations

Cited by in corpus (6)

• Bugs in Quantum Computing Platforms: An Empirical Study
• Diet Code Is Healthy: Simplifying Programs for Pre-trained Models of Code
• Syntax-Guided Program Reduction for Understanding Neural Code Intelligence Models
• Beyond Fidelity: Explaining Vulnerability Localization of Learning-based Detectors
• Code2Snapshot: Using Code Snapshots for Learning Representations of Source Code
• Study of Distractors in Neural Models of Code