It Takes Two to Tango: Towards Theory of AI's Mind

Abstract

Theory of Mind is the ability to attribute mental states (beliefs, intents, knowledge, perspectives, etc.) to others and recognize that these mental states may differ from one's own. Theory of Mind is critical to effective communication and to teams demonstrating higher collective performance. To effectively leverage the progress in Artificial Intelligence (AI) to make our lives more productive, it is important for humans and AI to work well together in a team. Traditionally, there has been much emphasis on research to make AI more accurate, and (to a lesser extent) on having it better understand human intentions, tendencies, beliefs, and contexts. The latter involves making AI more human-like and having it develop a theory of our minds. In this work, we argue that for human-AI teams to be effective, humans must also develop a theory of AI's mind (ToAIM) - get to know its strengths, weaknesses, beliefs, and quirks. We instantiate these ideas within the domain of Visual Question Answering (VQA). We find that using just a few examples (50), lay people can be trained to better predict responses and oncoming failures of a complex VQA model. We further evaluate the role existing explanation (or interpretability) modalities play in helping humans build ToAIM. Explainable AI has received considerable scientific and popular attention in recent times. Surprisingly, we find that having access to the model's internal states - its confidence in its top-k predictions, explicit or implicit attention maps which highlight regions in the image (and words in the question) the model is looking at (and listening to) while answering a question about an image - do not help people better predict its behavior.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• Recurrent Models of Visual Attention
• Object Detectors Emerge in Deep Scene CNNs
• Multiple Object Recognition with Visual Attention
• Generating Visual Explanations
• Attentive Explanations: Justifying Decisions and Pointing to the Evidence
• Reasoning About Pragmatics with Neural Listeners and Speakers
• Who is Mistaken?

Cited by in corpus (16)

• Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• Towards falsifiable interpretability research
• MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks
• Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction
• CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
• A Study on Multimodal and Interactive Explanations for Visual Question Answering
• Explainable Goal-Driven Agents and Robots -- A Comprehensive Review
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
• Designing for Democratization: Introducing Novices to Artificial Intelligence Via Maker Kits
• Modeling Conceptual Understanding in Image Reference Games
• Can you hear me $\text{now}$? Sensitive comparisons of human and machine perception
• Evaluating Theory of Mind in Question Answering
• Training Machine Learning Models by Regularizing their Explanations
• Improving Dialog Systems for Negotiation with Personality Modeling
• Generating and Evaluating Explanations of Attended and Error-Inducing Input Regions for VQA Models