Learning Program Embeddings to Propagate Feedback on Student Code

Abstract

Providing feedback, both assessing final work and giving hints to stuck students, is difficult for open-ended assignments in massive online classes which can range from thousands to millions of students. We introduce a neural network method to encode programs as a linear mapping from an embedded precondition space to an embedded postcondition space and propose an algorithm for feedback at scale using these linear maps as features. We apply our algorithm to assessments from the Code.org Hour of Code and Stanford University's CS1 course, where we propagate human comments on student assignments to orders of magnitude more submissions.

Accepted to International Conference on Machine Learning (ICML 2015)

References in corpus (4)

• Neural Turing Machines
• Learning to Discover Efficient Mathematical Identities
• Can recursive neural tensor networks learn logical reasoning?
• Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions

Cited by in corpus (26)

• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• Automated Grading and Feedback Tools for Programming Education: A Systematic Review
• A Convolutional Attention Network for Extreme Summarization of Source Code
• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks
• Learning Continuous Semantic Representations of Symbolic Expressions
• A Survey of Automated Programming Hint Generation -- The HINTS Framework
• Toward Semi-Automatic Misconception Discovery Using Code Embeddings
• SnapCheck: Automated Testing for Snap Programs
• Latent Skill Embedding for Personalized Lesson Sequence Recommendation
• Neutaint: Efficient Dynamic Taint Analysis with Neural Networks
• Automatic Assessment of the Design Quality of Python Programs with Personalized Feedback
• Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference
• Deep Learning for Bug-Localization in Student Programs
• Play to Grade: Testing Coding Games as Classifying Markov Decision Process
• Teaching Temporal Logics to Neural Networks
• Neural Program Synthesis with a Differentiable Fixer
• ast2vec: Utilizing Recursive Neural Encodings of Python Programs
• Improving the Robustness to Data Inconsistency between Training and Testing for Code Completion by Hierarchical Language Model
• Improve Language Modelling for Code Completion through Statement Level Language Model based on Statement Embedding Generated by BiLSTM
• Enabling Open-World Specification Mining via Unsupervised Learning
• Synthesizing Tasks for Block-based Programming
• Improving Automatic Source Code Summarization via Deep Reinforcement Learning
• Context-Aware Attentive Knowledge Tracing
• Semi-Supervised Verified Feedback Generation
• Math Operation Embeddings for Open-ended Solution Analysis and Feedback