Learnable latent embeddings for joint behavioral and neural analysis

Abstract

Mapping behavioral actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioral data increases, there is growing interest in modeling neural dynamics during adaptive behaviors to probe neural representations. In particular, neural latent embeddings can reveal underlying correlates of behavior, yet, we lack non-linear techniques that can explicitly and flexibly leverage joint behavior and neural data. Here, we fill this gap with a novel method, CEBRA, that jointly uses behavioral and neural data in a hypothesis- or discovery-driven manner to produce consistent, high-performance latent spaces. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks, and in simple or complex behaviors across species. It allows for single and multi-session datasets to be leveraged for hypothesis testing or can be used label-free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, and rapid, high-accuracy decoding of natural movies from visual cortex.

Website: cebra.ai

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• The NumPy array: a structure for efficient numerical computation
• Large-scale neural recordings call for new insights to link brain and behavior
• Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity
• Strong and weak principles of neural dimension reduction
• Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

Cited by in corpus (8)

• SuperAnimal pretrained pose estimation models for behavioral analysis
• Interpretable statistical representations of neural population dynamics and geometry
• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
• Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systems
• Energy-information trade-off makes the cortical critical power law the optimal coding
• Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE
• Predictive variational autoencoder for learning robust representations of time-series data
• Characterizing Neural Manifolds' Properties and Curvatures using Normalizing Flows