A Schematic Representation for Cognitive Tool-Using Agents

Abstract

In artificial intelligence (AI) research, embodied systems have received increasing attention since the 1990s. How to bridge the gap between raw sensorimotor data and symbolic representation in a robotic agent is still an open question. The research described in this document is inspired by theories in cognitive science, such as concept theory and embodied realism, as well as work in robotics and AI. The general goal of this research is to build a system capable of acquiring and maintaining semantic knowledge for higher-level reasoning, in particular reasoning about the use of tools, from the embodied experience of a cognitive agent in a simulated environment or in the real world. This research addresses cognitive theories of embodiment, the design of a general computational architecture, and the design and implementation of AI techniques for solving tool-using problems. One of the major contributions of this research is to provide a computational architecture for an embodied agent that can capture semantic relations from its interactions with the world, sufficient to support effective tool use both in short-term predictions and plan generation. As a result, we have implemented an example of this architecture in an Action Schema Generator, or ASG, which can automatically generate production rules and symbolic representations from a simulated agent’s embodied experience without losing the capability of transferring the knowledge backwards to its original numerical sensorimotor format. We have developed pragmatic methods to evaluate the performance of ASG, at the component level and the system level, in simulated and real scenarios, for tasks with and without tools. We also have compared our design with other robotics and cognitive architectures, including behavior-based robotics, Neuroevolution, and psychologically inspired architectures. We believe that our work can provide a general foundation for embodied agents, and should be useful in future research.