Applying a Rubric Development Cycle for Assessment in Higher Education: An Evidence-based Case Study of a Science Communication Module

Abstract

Although empirical studies on the use of rubrics have been undertaken in a wide range of disciplines and for various purposes in higher education, surprisingly little attention has been paid to the process of rubric development to establish the quality of rubrics, especially in the assessment of science communication. In the context of a new science undergraduate module where this study is undertaken, it is essential to develop and validate the rubric that can assess the extent to which students are able to attain the learning outcomes to ensure that students’ learning outcomes are accurately and consistently assessed. This study presents a validation study of a task-specific assessment rubric developed for a communication module for science undergraduates using an embedded mixed-method design. The results of the Many-facet Rasch Model analysis indicate that the rubric appears to be functioning well, with the raters, items and rating scale functioning as intended by the model; however, the scale descriptors need fine-tuning. The qualitative analysis of tutors’ feedback also supports revision and recommends refinement of the descriptors. The results from the surveys show that students found the rubric useful in helping them understand their achievement levels and enhance their writing performance. This study not only provides direction for future revisions of the rubric, but also confirms the importance of a robust process of developing and validating rubrics in higher education using a rubric development cycle, especially when a large cohort of students and multiple raters are involved.