Automatic diagnosis of student programs in programming learning environments

Abstract

This paper describes a method to automate the diagnosis of students' programming errors in programming learning environments. In order to recognize correct students' programs as well as to identify errors in incorrect student programs, programs are represented using an improved dependence graph representation. The student program is compared with a specimen program (also called a model program) at the semantic level after both are standardized by program transformations. The method is implemented using Smalltalk in SIPLeS-II, an automatic program diagnosis system for Samlltalk programming learning environments. The system has been tested on approximately 330 student programs for various tasks. Experimental results show that, using the method, semantic errors in a student program can be identified rigorously and safely. Semantics-preserving variations in a student program can be eliminated or accommodated. The tests also show that the system can identify a wide range of errors as well as produce indications of the corrections needed. This method is essential for the development of programming learning environments. The techniques of the improved program dependence graph representation, program standardization by transformations, and semantic level program comparison are also useful in other research fields including program understanding and software maintenance.