Toward Imitating Visual Attention of Experts in Software Development Tasks
arXiv:1903.06320
Abstract
Expert programmers' eye-movements during source code reading are valuable sources that are considered to be associated with their domain expertise. We advocate a vision of new intelligent systems incorporating expertise of experts for software development tasks, such as issue localization, comment generation, and code generation. We present a conceptual framework of neural autonomous agents based on imitation learning (IL), which enables agents to mimic the visual attention of an expert via his/her eye movement. In this framework, an autonomous agent is constructed as a context-based attention model that consists of encoder/decoder network and trained with state-action sequences generated by an experts' demonstration. Challenges to implement an IL-based autonomous agent specialized for software development task are discussed in this paper.
arXiv comment4 pages, EMIP 2019
References in corpus (6)
- Generative Adversarial Imitation Learning
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
- Query-Efficient Imitation Learning for End-to-End Autonomous Driving
- A Syntactic Neural Model for General-Purpose Code Generation
- Learning to Reason: End-to-End Module Networks for Visual Question Answering
- Learning to Generate Corrective Patches using Neural Machine Translation