DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks

Abstract

Non-contact video-based physiological measurement has many applications in health care and human-computer interaction. Practical applications require measurements to be accurate even in the presence of large head rotations. We propose the first end-to-end system for video-based measurement of heart and breathing rate using a deep convolutional network. The system features a new motion representation based on a skin reflection model and a new attention mechanism using appearance information to guide motion estimation, both of which enable robust measurement under heterogeneous lighting and major motions. Our approach significantly outperforms all current state-of-the-art methods on both RGB and infrared video datasets. Furthermore, it allows spatial-temporal distributions of physiological signals to be visualized via the attention mechanism.

arXiv commentAccepted paper at ECCV 2018. 16 pages, 3 figures, supplementary materials in the ancillary files

References in corpus (9)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• ADADELTA: An Adaptive Learning Rate Method
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Action Recognition using Visual Attention
• Unsupervised Learning for Physical Interaction through Video Prediction
• Describing Videos by Exploiting Temporal Structure
• Learning Spatiotemporal Features with 3D Convolutional Networks
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Two-stream Flow-guided Convolutional Attention Networks for Action Recognition

Cited by in corpus (3)

• Real-time Webcam Heart-Rate and Variability Estimation with Clean Ground Truth for Evaluation
• Diverse R-PPG: Camera-Based Heart Rate Estimation for Diverse Subject Skin-Tones and Scenes
• Learning Higher-Order Dynamics in Video-Based Cardiac Measurement