Can an instructional video increase the quality of English teachers’ assessment of learner essays?

Teacher judgments of student achievement influence how effectively students learn and how teachers organize their lessons. Teachers’ diagnostic competence is therefore an important field of research for subject-specific teaching and learning. This study investigates how pre-service English teachers assess complex learner essays using assessment rubrics. In particular, it explores whether instructional videos are effective in minimizing distortion effects in essay assessment by raising participants’ awareness about them. English pre-service teachers (N = 81) in Switzerland and Germany assessed four argumentative essays of varying lexical and overall quality. Prior to the assessment task, the treatment group saw a ten-minute video about how to assess vocabulary, while the control group watched a video about general aspects of assessment. In their judgments, pre-service teachers recognized higher/lower overall quality and higher/lower lexical quality. More importantly, the results showed significant effects of the variation in lexical quality on other text characteristics, indicating halo effects. These effects were similar in both groups, suggesting that the video instruction on its own was no safeguard against distortion effects in essay assessment. Implications for teacher education and further research are discussed.