Low-power, Continuous Remote Behavioral Localization with Event Cameras

Abstract

Researchers in natural science need reliable methods for quantifying animal behavior. Recently, numerous computer vision methods emerged to automate the process. However, observing wild species at remote locations remains a challenging task due to difficult lighting conditions and constraints on power supply and data storage. Event cameras offer unique advantages for battery-dependent remote monitoring due to their low power consumption and high dynamic range capabilities. We use this novel sensor to quantify a behavior in Chinstrap penguins called ecstatic display. We formulate the problem as a temporal action detection task, determining the start and end times of the behavior. For this purpose, we recorded a colony of breeding penguins in Antarctica for several weeks and labeled event data on 16 nests. The developed method consists of a generator of candidate time intervals (proposals) and a classifier of the actions within them. The experiments show that the event cameras' natural response to motion is effective for continuous behavior monitoring and detection, reaching a mean average precision (mAP) of 58% (which increases to 63% in good weather conditions). The results also demonstrate the robustness against various lighting conditions contained in the challenging dataset. The low-power capabilities of the event camera allow it to record significantly longer than with a conventional camera. This work pioneers the use of event cameras for remote wildlife observation, opening new interdisciplinary opportunities. https://tub-rip.github.io/eventpenguins/

arXiv comment13 pages, 8 figures, 12 tables, Project page: https://tub-rip.github.io/eventpenguins/

References in corpus (8)

• Event-based Vision: A Survey
• Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
• Single Shot Temporal Action Detection
• End-to-end Temporal Action Detection with Transformer
• Learning to Detect Objects with a 1 Megapixel Event Camera
• ATRW: A Benchmark for Amur Tiger Re-identification in the Wild
• A Large Scale Event-based Detection Dataset for Automotive
• The iWildCam 2018 Challenge Dataset

Cited by in corpus (2)

• Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
• Characterizing and Modeling AI-Driven Animal Ecology Studies at the Edge