A Comparison of Knowledge Acquisition Methods for the Elicitation of Procedural Mental Models
No Thumbnail Available
Files
Date
2002-12-19
Authors
Journal Title
Series/Report No.
Journal ISSN
Volume Title
Publisher
Abstract
This study compared the effectiveness of two different knowledge acquisition methods in terms of their ability to measure the structure of a student's procedural mental model of computer programming as it evolved in a classroom setting over the course of a semester. Ratings and backward thinking tasks were used to estimate the structure of students' procedural mental models at several points throughout an introductory programming course. For both methods, comparisons between the student's procedural mental model and a prototype model (i.e., the instructor's conceptual model of the knowledge provided in the course) were used to identify how the procedural tasks and the relationships among tasks are represented in the student's mental model. The values from the ratings task were input into the Pathfinder scaling algorithm, which compares the structural similarity between a prototype model and the student's mental model, to generate an index of similarity. Additionally, the Pathfinder scaling algorithm measured the consistency of the student's rating process via the coherence index. The steps recorded during the backward thinking task were compared to the prototype model to identify the number of steps out of sequence. The first hypothesis was that there would be a positive correlation between the index of similarity and the project score. However, this hypothesis was only partially supported, as the correlation between the index of similarity and project score was significant only at test time three. The second hypothesis was that as the student completed each successive project, the index of similarity would increase over the course of the semester. This hypothesis was not supported because the index of similarity never increased beyond the value obtained at test time one. The third hypothesis was that as the student completed each successive project, the coherence index would increase. This hypothesis was not supported, as no main effect of test time on the coherence index was found. The fourth hypothesis was that there would be a negative correlation between the project score and the number of steps out of sequence. The number of steps out of sequence was excluded from further analysis due to low inter-rater reliability. Therefore this hypothesis cannot be evaluated. Lastly, it was hypothesized that the index of similarity would correlate more positively with student's performance than the number of steps out of sequence. Again, without the number of steps out of sequence, this hypothesis could not be fully evaluated. However, for project three, the index of similarity was significantly positively correlated with the project score.
Description
Keywords
knowledge acquisition, mental models, procedural knowledge, backward thinking, Pathfinder
Citation
Degree
PhD
Discipline
Psychology
