Interactive Inference: A Neuromorphic Theory of Human-Computer Interaction

Abstract

Neuromorphic Human-Computer Interaction (HCI) is a theoretical approach to designing better user experiences (UX) motivated by advances in the understanding of the neurophysiology of the brain. Inspired by the neuroscientific theory of Active Inference, Interactive Inference is a first example of such an approach. It offers a simplified interpretation of Active Inference that allows designers to more readily apply this theory to design and evaluation. The basic premise in Interactive Inference is that the user predicts a result prior to performing a task. User behaviour is modeled as Bayesian inference on progress and goal distributions that predicts the next action. The difference between the observed result and the prediction is what is processed by the brain. This discrepancy between goal and progress distributions, or Bayesian surprise, can be modeled as a simple mean square error of the signal-to-noise ratio (SNR) of a task. The problem is that the user's capacity to process Bayesian surprise follows the logarithm of this SNR. This means terminal execution errors rise quickly once average capacity is exceeded. Our model allows the quantitative analysis of performance and terminal error using one framework that can provide real-time estimates of the mental load of users that needs to be minimized by design. We show how three basic laws of HCI, Hick's Law, Fitts' Law and the Power Law can be expressed using our model. We then test the validity of the model by empirically measuring how well it predicts human performance and terminal error in a car following task. Results suggest that driver processing capacity is indeed a logarithmic function of the SNR of the distance to a lead car, and that terminal error follows the residual surprise predicted by a KL-divergence model. This result provides initial evidence that Interactive Inference can be useful as a new theoretical design tool.

19 pages, 7 figures, 1 table, 37 mathematical formulas, in press