The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

Abstract

Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. To help accelerate behavioral studies, the CalMS21 dataset provides benchmarks to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabeled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labeled and unlabeled tracking data, as well as being able to generalize to new settings.

arXiv commentNeurIPS2021 Datasets & Benchmarks. Dataset: https://data.caltech.edu/records/1991, Website: https://sites.google.com/view/computational-behavior/our-datasets/calms21-dataset

References in corpus (13)

• Scikit-learn: Machine Learning in Python
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Deep Visual Domain Adaptation: A Survey
• YouTube-8M: A Large-Scale Video Classification Benchmark
• When Does Label Smoothing Help?
• A Survey on Multi-Task Learning
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization
• Learning recurrent representations for hierarchical behavior modeling
• MineRL: A Large-Scale Dataset of Minecraft Demonstrations

Cited by in corpus (3)

• Of Mice and Mates: Automated Classification and Modelling of Mouse Behaviour in Groups using a Single Model across Cages
• Unsupervised Learning of Neurosymbolic Encoders
• Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis