A method to predict interrater agreement in writing assessment based on raters' individual differences

Roberto Jaunez, Jorge Villalon, Gonzalo Munoz, Gabriela Baez

Resultado de la investigación: Capítulo del libro/informe/acta de congresoContribución a la conferenciarevisión exhaustiva

Resumen

Writing has been increasingly assessed within educational scenarios. However, assessing writing is a highly subjective task, making the grading of such tests a costly process that demands several graders for calculating interrater agreement to ensure reliability. A drawback of this approach is that interrater agreement does not explain the source of markers' differences; therefore, when agreement is low, the marking process must be repeated, and markers retrained. This article proposes a method to predict agreement, based on markers' individual differences using item response theory. Results showed that the method accurately predicts agreement even with partial evidence.

Idioma originalInglés
Título de la publicación alojadaProceedings - IEEE 19th International Conference on Advanced Learning Technologies, ICALT 2019
EditoresMaiga Chang, Demetrios G Sampson, Ronghuai Huang, Alex Sandro Gomes, Nian-Shing Chen, Ig Ibert Bittencourt, Kinshuk Kinshuk, Diego Dermeval, Ibsen Mateus Bittencourt
EditorialInstitute of Electrical and Electronics Engineers Inc.
Páginas255-257
Número de páginas3
ISBN (versión digital)9781728134857
DOI
EstadoPublicada - jul. 2019
Evento19th IEEE International Conference on Advanced Learning Technologies, ICALT 2019 - Maceio, Brasil
Duración: 15 jul. 201918 jul. 2019

Serie de la publicación

NombreProceedings - IEEE 19th International Conference on Advanced Learning Technologies, ICALT 2019

Conferencia

Conferencia19th IEEE International Conference on Advanced Learning Technologies, ICALT 2019
País/TerritorioBrasil
CiudadMaceio
Período15/07/1918/07/19

Huella

Profundice en los temas de investigación de 'A method to predict interrater agreement in writing assessment based on raters' individual differences'. En conjunto forman una huella única.

Citar esto