User-defined difficulty levels for automated question generation

Abstract

We propose a difficulty model for generating questions across formal domains according to the difficulty level provided by the user. Our model is interactive and adaptive to user input. The model uses predefined factors for measuring the difficulty and a user defines the difficulty level by ordering these factors. We use lexicographical ordering to compare the difficulty of questions based on a user-defined ordering of factors and a concomitant algorithm for handling these factors. Further, we provide a feature called scenario guidance, which allows users to change the scenario at run time. We develop a software using the proposed model, which generates new questions according to a user-defined difficulty level. In order to evaluate the proposed framework, we conducted a pilot test of the software, in which teachers generate questions according to their chosen desired input including the difficulty level. The results show that the system is effective, helpful and robust. Overall, the framework shows promising benefits for teachers and organizations involved in setting questions for standardized tests.