High Speed Human Action Recognition using a Photonic Reservoir Computer
arXiv:2305.15283 · doi:10.1016/j.neunet.2023.06.014
Abstract
The recognition of human actions in videos is one of the most active research fields in computer vision. The canonical approach consists in a more or less complex preprocessing stages of the raw video data, followed by a relatively simple classification algorithm. Here we address recognition of human actions using the reservoir computing algorithm, which allows us to focus on the classifier stage. We introduce a new training method for the reservoir computer, based on "Timesteps Of Interest", which combines in a simple way short and long time scales. We study the performance of this algorithm using both numerical simulations and a photonic implementation based on a single non-linear node and a delay line on the well known KTH dataset. We solve the task with high accuracy and speed, to the point of allowing for processing multiple video streams in real time. The present work is thus an important step towards developing efficient dedicated hardware for video processing.
arXiv comment(article in press)
References in corpus (9)
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- Multiple Object Recognition with Visual Attention
- All-optical Reservoir Computing
- Reinforcement Learning in a large scale photonic Recurrent Neural Network
- High performance photonic reservoir computer based on a coherently driven passive cavity
- Human action recognition with a large-scale brain-inspired photonic computer
- Reservoir computing based on a silicon microring and time multiplexing for binary and analog operations
- Large-scale spatiotemporal photonic reservoir computer for image classification
- Parallel photonic reservoir computing using frequency multiplexing of neurons