New ASIST/ToMCAT Grant

ToMCAT. A collaboration between the Information School (INFO), Computer Science (CS), and Family Studies and Human Development (FSHD) has been awarded a large grant to develop a theory of mind-based cognitive architecture for teams (ToMCAT). The grant ($7.5M, for 48 months) is part of the new DARPA Artificial Social Intelligence for Successful Teams (ASIST)program. The PI/Co-PIs collaborating on this project are: Adarsh Pyarelal (PI),Kobus Barnard,Emily Butler,Clayton Morrison,Rebecca Sharp,Mihai Surdeanu, andMarco Antonio Valenzuela-Escarcega. Data collection for the project will take place in the Lang Laboratory, housed in the Frances McClelland Institute for Children, Youth and Families in the Norton School of Family & Consumer Science.

The goal of the project is to build artificially intelligent agents that understand both the social and goal-oriented aspects of teams in mission-like scenarios (e.g., search-and-rescue missions), and are able to reason about possible interventions to steer the team. The agent, ToMCAT, needs to model human players' affect and beliefs about the situation and about each other’s affect and beliefs (theory of mind). We will ground this work in extensive measurements of humans interacting in small teams, that will include audio, video, eye tracking, electrocardiography (EKG), electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), and self report. The participants will execute missions within a Minecraft environment with one, two, three, or four human players interacting with the ToMCAT agent.

Research areas. One unique aspect of this project is that we will use simultaneous EEG and fNIRS brain recording from all human team members to further our understanding of social coordination in teams. We expect the series of experiments will provide a large amount of unique data. ToMCAT's evolving theories of mind will be implemented using dynamic Bayesian networks interacting with latent low-level data representation provided by neural networks. In addition, we will need to understand dialogue as indicative of affect, plans, and mission goals. Finally, ToMCAT will need to both understand team plans and also create its own plans.

Further information is available on the project web site ml4ai.github.io/tomcat. This project started Nov 1, 2019. As we move forward, we will update this website regularly.