Predicting Daily Activities From Egocentric Images Using Deep Learning

Abstract

We present a method to analyze images taken from a passive egocentric wearable camera along with the contextual information, such as time and day of week, to learn and predict everyday activities of an individual. We collected a dataset of 40,103 egocentric images over a 6 month period with 19 activity classes and demonstrate the benefit of state-of-the-art deep learning techniques for learning and predicting daily activities. Classification is conducted using a Convolutional Neural Network (CNN) with a classification method we introduce called a late fusion ensemble. This late fusion ensemble incorporates relevant contextual information and increases our classification accuracy. Our technique achieves an overall accuracy of 83.07% in predicting a person's activity across the 19 activity classes. We also demonstrate some promising results from two additional users by fine-tuning the classifier with one day of training data.

8 pages

References in corpus (2)

• Improving neural networks by preventing co-adaptation of feature detectors
• Caffe: Convolutional Architecture for Fast Feature Embedding

Cited by in corpus (9)

• SR-Clustering: Semantic Regularized Clustering for Egocentric Photo Streams Segmentation
• Video-based Human Action Recognition using Deep Learning: A Review
• RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
• Batch-based Activity Recognition from Egocentric Photo-Streams Revisited
• DeepDiary: Automatic Caption Generation for Lifelogging Image Streams
• Recognizing Activities of Daily Living from Egocentric Images
• On the Role of Event Boundaries in Egocentric Activity Recognition from Photostreams
• Personalized Image-based User Authentication using Wearable Cameras
• PAL: Intelligence Augmentation using Egocentric Visual Context Detection