Computing a human-like reaction time metric from stable recurrent vision models

Abstract

The meteoric rise in the adoption of deep neural networks as computational models of vision has inspired efforts to "align" these models with humans. One dimension of interest for alignment includes behavioral choices, but moving beyond characterizing choice patterns to capturing temporal aspects of visual decision-making has been challenging. Here, we sketch a general-purpose methodology to construct computational accounts of reaction times from a stimulus-computable, task-optimized model. Specifically, we introduce a novel metric leveraging insights from subjective logic theory summarizing evidence accumulation in recurrent vision models. We demonstrate that our metric aligns with patterns of human reaction times for stimulus manipulations across four disparate visual decision-making tasks spanning perceptual grouping, mental simulation, and scene categorization. This work paves the way for exploring the temporal alignment of model and human visual strategies in the context of various other cognitive tasks toward generating testable hypotheses for neuroscience. Links to the code and data can be found on the project page: https://serre-lab.github.io/rnn_rts_site.

arXiv commentPublished at NeurIPS 2023

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Axiomatic Attribution for Deep Networks
• Evidential Deep Learning to Quantify Classification Uncertainty
• Adaptive Computation Time for Recurrent Neural Networks
• Biologically inspired protection of deep networks from adversarial attacks
• Learning model-based planning from scratch
• Deep Sequential Neural Network
• Image Features Influence Reaction Time: A Learned Probabilistic Perceptual Model for Saccade Latency
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation