An Inductive Framework for Affect Recognition and Expression in Interactive Learning Environments

Abstract

Recent years have seen a growing recognition of the importance of affective reasoning in human-computer interaction. Because affect plays an important role in cognitive functions, such as perception and decision-making, the prospect of modeling user affect and enabling interactive systems to respond appropriately holds much appeal for a broad range of applications. Affective reasoning is particularly promising for educational applications because of the strong connections between affect and learning. If it were possible to accurately detect frustration, monitor changes in efficacy, and predict students’ affective states, interactive learning environments could more effectively tailor problem-solving episodes. However, constructing computational models of affect recognition and affect expression is challenging because of the need to devise solutions that are accurate, efficient, and capable of making early predictions. To this end we propose CARE, an inductive framework for affect recognition and expression. CARE learns models of affect from observation of human-computer and human-human interaction. First, in training sessions, users perform a series of tasks in interactive environments while CARE monitors reports of users’ affective experiences. In addition, CARE monitors user actions, world state, and physiological responses. Second, CARE induces models of affect from observed data with machine learning techniques that include decision trees, naive Bayes classifiers, support vector machines, Bayesian networks, and n-grams. Third, at runtime, CARE-induced models monitor user actions, world state, and physiological responses to predict user affective states. In a series of studies involving more than four hundred subjects, the CARE framework has successfully been used to perform a number of affect prediction tasks, including emotional state prediction, self-efficacy, and metacognitive monitoring prediction. It has also been used to induce models of empathy for virtual agents in interactive learning environments. Results suggest that CARE-induced affect models satisfy the real-time requirements of interactive systems and provide a solid foundation for empirically informed affective reasoning.