Deterministic Step-Level Verification of Calculus Solutions with Simulated Feedback Control
- Roman Yavich
- Vladimir Rotkin
Abstract
Generative artificial intelligence complicates the interpretation of take-home mathematics assignments, while checking final answers alone provides limited diagnostic information. We develop and evaluate a computational assessment architecture that verifies explicit solution steps against the learner’s stated intermediate results. An error catalogue screens task variants and supplies cautious diagnostic labels; it does not determine correctness. A language-model component is restricted to wording feedback whose structured claims are checked against the verified solution graph. Three simulation studies, designed as a technical-validation stage with exactly known ground truth, used 33,986 generated answers, 19,883 simulated solutions, and simulated feedback distortions for three calculus templates from a mathematics course for economics students. Under naive parameter selection, 33.9% of variants allowed at least one catalogue error to produce the correct final answer. In symbolic mode, all 8,000 correct solutions across eight implemented methods were accepted, and every nondegenerate injected error with an explicit basis was located. In 294 of the 2,944 solutions with one injected out-of-catalogue slip (10.0%), the final answer remained correct, and step-level verification identified the erroneous step in every such case. The feedback filter rejected all modelled distortions of checkable claims, although unstructured explanations remained outside its guarantee. The contribution is a formal separation of verification, diagnosis, and feedback generation, supported by controlled computational evidence. The findings characterise the architecture within the implemented step language and define what a subsequent field study must measure; classroom effectiveness, resistance to unauthorised assistance and the behaviour of an actual language model remain to be examined.
- Full Text:
PDF
- DOI:10.5539/hes.v16n4p869
Index
Contact
- Sherry LinEditorial Assistant
- hes@ccsenet.org