Abstract

This research addresses the critical gap in the automated assessment of open-ended mathematical responses by developing a novel diagnostic technique that integrates multidimensional item response theory with real-time analysis of misconceptions and adaptive feedback generation. Unlike existing automated assessment systems limited to multiple-choice formats or binary scoring, this study pioneers the automated evaluation of subjective mathematical work in geometry and algebra while simultaneously diagnosing specific misconception patterns. The research analyzed 517 seventh-grade students’ responses across four Thai regions to establish empirically grounded cutoff points for five proficiency levels in two dimensions: mathematical processes (-2.37, -0.16, 0.89, 1.06) and conceptual structures (-2.69, 0.24, 0.46, 1.03). The innovative contribution lies in categorizing misconceptions into four distinct types (overgeneralization, defective mathematical understanding, mistranslation, and limited conception) and linking each to five differentiated feedback modes, creating a pedagogically-driven automated response system. The multidimensional model demonstrated superior psychometric properties compared to unidimensional approaches, with reliability coefficients of 0.83 and 0.80 for the respective dimensions. Implementation within the eMAT-Testing platform enabled real-time diagnostic capability, processing subjective responses containing mathematical expressions, and providing targeted feedback based on identified misconception patterns. This breakthrough enables the use of a scalable formative assessment technique previously requiring human expertise, with system evaluation showing the highest appropriateness ratings for user interaction (x̄ = 5.00, SD = 0.00) and responsibility aspects (x̄ = 4.89, SD = 0.58). The technique’s novelty extends the application of construct modeling theory to automated assessment practice, demonstrating how sophisticated psychometric frameworks can maintain rigor while delivering immediate, pedagogically-meaningful diagnostic information for differentiated instruction.

This work is licensed under a Creative Commons Attribution 4.0 License.