Reinforcement learning for adaptive theory of mind in the sigma cognitive architecture

Pynadath, D. V., Rosenbloom, P. S. and Marsella, S. C. (2014) Reinforcement learning for adaptive theory of mind in the sigma cognitive architecture. In: Goertzel, B., Orseau, L. and Snaider, J. (eds.) Artificial General Intelligence: 7th International Conference, AGI 2014, Quebec City, QC, Canada, August 1-4, 2014. Proceedings. Springer, pp. 143-154. ISBN 9783319092737 (doi: 10.1007/978-3-319-09274-4_14)

Full text not currently available from Enlighten.


One of the most common applications of human intelligence is social interaction, where people must make effective decisions despite uncertainty about the potential behavior of others around them. Reinforcement learning (RL) provides one method for agents to acquire knowledge about such interactions. We investigate different methods of multiagent reinforcement learning within the Sigma cognitive architecture. We leverage Sigma’s architectural mechanism for gradient descent to realize four different approaches to multiagent learning: (1) with no explicit model of the other agent, (2) with a model of the other agent as following an unknown stationary policy, (3) with prior knowledge of the other agent’s possible reward functions, and (4) through inverse reinforcement learning (IRL) of the other agent’s reward function. While the first three variations re-create existing approaches from the literature, the fourth represents a novel combination of RL and IRL for social decision-making. We show how all four styles of adaptive Theory of Mind are realized through Sigma’s same gradient descent algorithm, and we illustrate their behavior within an abstract negotiation task.

Item Type:Book Sections
Glasgow Author(s) Enlighten ID:Marsella, Professor Stacy
Authors: Pynadath, D. V., Rosenbloom, P. S., and Marsella, S. C.
College/School:College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience

University Staff: Request a correction | Enlighten Editors: Update this record