Multimodal Pedagogical Planning for 3D Learning Environments - A Unified Framework

Show simple item record

dc.contributor.advisor Dr. Robert St. Amant, Committee Member en_US
dc.contributor.advisor Dr. Munindar P. Singh, Committee Member en_US
dc.contributor.advisor Dr. R. Michael Young, Committee Member en_US
dc.contributor.advisor Dr. James C. Lester, Committee Chair en_US Zhang, Wei en_US 2010-04-02T18:58:48Z 2010-04-02T18:58:48Z 2004-04-19 en_US
dc.identifier.other etd-03312004-220912 en_US
dc.description.abstract Pedagogical planning lies at the heart of knowledge-based learning environments. In recent years, multimodality and authoring have become key issues in the creating of learning environments. The purpose of this research has been to design, implement, and evaluate a multimodal pedagogical planning system and a multimodal pedagogical authoring system. First, we designed and implemented a multimodal pedagogical planning system for 3D learning environments. In 3D learning environments, the student can cooperate with an animated agent to solve a problem or study a new concept. The agent uses natural language narration accompanied by synchronized gestures to communicate with the student. 3D animation and camera motions are also used to offer the student a vivid and friendly learning environment. In order to control the agent's utterances, the agent's gestures and 3D multimedia presentation in the 3D learning environment, we designed a multimodal pedagogical planning system that uses a coordinated distributed planning structure. The multimodal pedagogical planning system contains a Pedagogical Planner which is a global planner, and three coordinated distributed planners: an Agent Utterance Planner, an Agent Gesture Planner, and a Cameral and Animation Planner. The planners synchronize the agent's verbal and physical behaviors on phrase boundaries. We implemented the multimodal pedagogical planning system in the PhysViz 3D learning environment for the circuit experiments. Second, we designed and implemented a multimodal pedagogical authoring system based on the multimodal pedagogical planning architecture. One of the most difficult problems faced by ITS research is authoring. Because of their complexity, ITSs are notoriously difficult to author. However, because of the modularity of the multimodal pedagogical planning architecture we developed, we hypothesized that our model of multimodal communication designed above can serve as the basis for an ITS authoring environment. The multimodal pedagogical authoring system can be used to author the pedagogical plans, including plans that describe the agent's utterance, agent's gesture, camera motions and animations. We empirically evaluated the authoring architecture in conjunction with the PhysViz 3D learning environment by studying non-technical subjects' ability to modify learning activities for PhysViz. In the informal evaluation, 10 subjects were invited to use our multimodal pedagogical authoring system and answer a questionnaire. The results of this questionnaire suggest that most subjects without advanced knowledge of Computer Science can build 3D learning environments with the help of our authoring system. en_US
dc.rights I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to NC State University or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report. en_US
dc.subject Multimodal Pedagogical Planning en_US
dc.subject Intelligent Tutoring Systems en_US
dc.subject ITS Authoring Systems en_US
dc.title Multimodal Pedagogical Planning for 3D Learning Environments - A Unified Framework en_US PhD en_US dissertation en_US Computer Science en_US

Files in this item

Files Size Format View
etd.pdf 2.071Mb PDF View/Open

This item appears in the following Collection(s)

Show simple item record