Strategic Deception in Agents

Abstract

Despite its negative ethical connotations, deception is a useful tool for human social interaction, and plays an important role in the process of creating the stories that pervade our popular culture. In this paper, we describe the deception planner, an implementation of a model of strategic deception. Strategic deception is deception performed in order to achieve or enable some higher goal, as opposed to deception that is performed for the sake of deceiving, or for an unstated purpose.

Given a model of a deceiver holding ulterior goals, a model of the goals and abilities of a target agent to be deceived, and a model of the relevant pieces of the world, the deception planner generates a set of statements about the current world state which may be either true or false. Those statements are communicated to the target agent, which updates its world state to reflect this new information. The target then performs planning to achieve its own goals, with no knowledge of the deceiver's goals. If the deception planner generates an successful set of statements, the target agent will create a plan that achieves the deceiver's ulterior goals despite no knowledge of those goals.

To find the set of statements that will generate this desired behavior from the target agent, the deception planner models the target agent's planning process. The deception planner searches for a plan that achieves the target agent's goals as well as the deceiver's ulterior goals. When such a plan is found, it is labeled the candidate plan, and the deceiver gives the target agent enough (dis)information so that, given the target agent's knowledge of the world, she can generate that plan. The candidate plan may depend on lies, chosen by the deception planner. The planner ensures that any lies told are not discovered before the target executes enough of the plan to achieve the ulterior goals.

Once a candidate plan has been found, the deception planner the finds and counters competing plans. Competing plans are plan that achieve the target's goals but not the deceiver's goals, but are of equivalent or better quality than the candidate plan according to some metric shared by the target and deceiving agent. Because they are of the same or better quality, the target agent may choose a competing plan instead of the candidate plan. A competing plan is countered by undermining through lying a belief that is necessary for that plan to be executable.

Although the deception planner fits within a body of work on agent deception, the goal of this algorithm is unique in its focus on causing a target agent to act in order to achieve a deceiver's goals, and in its utilization of a model of the goals and planning abilities of the target agent to that end.