KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints

Abstract

Computer vision/deep learning-based 3D human pose estimation methods aim to localize human joints from images and videos. Pose representation is normally limited to 3D joint positional/translational degrees of freedom (3DOFs), however, a further three rotational DOFs (6DOFs) are required for many potential biomechanical applications. Positional DOFs are insufficient to analytically solve for joint rotational DOFs in a 3D human skeletal model. Therefore, we propose a temporal inverse kinematics (IK) optimization technique to infer joint orientations throughout a biomechanically informed, and subject-specific kinematic chain. For this, we prescribe link directions from a position-based 3D pose estimate. Sequential least squares quadratic programming is used to solve a minimization problem that involves both frame-based pose terms, and a temporal term. The solution space is constrained using joint DOFs, and ranges of motion (ROMs). We generate 3D pose motion sequences to assess the IK approach both for general accuracy, and accuracy in boundary cases. Our temporal algorithm achieves 6DOF pose estimates with low Mean Per Joint Angular Separation (MPJAS) errors (3.7°/joint overall, & 1.6°/joint for lower limbs). With frame-by-frame IK we obtain low errors in the case of bent elbows and knees, however, motion sequences with phases of extended/straight limbs results in ambiguity in twist angle. With temporal IK, we reduce ambiguity for these poses, resulting in lower average errors.

arXiv commentProject page: https://kevgildea.github.io/KinePose/

References in corpus (61)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Categorical Reparameterization with Gumbel-Softmax
• In Defense of the Triplet Loss for Person Re-Identification
• Squeeze-and-Excitation Networks
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Learning-Based View Synthesis for Light Field Cameras
• Learning Representations by Maximizing Mutual Information Across Views
• CrowdHuman: A Benchmark for Detecting Human in a Crowd
• Data Augmentation using Random Image Cropping and Patching for Deep CNNs
• Deep Clustering for Unsupervised Learning of Visual Features
• Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods
• Do ImageNet Classifiers Generalize to ImageNet?
• Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition
• InfoVAE: Information Maximizing Variational Autoencoders
• Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
• YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
• Deep Drone Racing: From Simulation to Reality with Domain Randomization
• Lung and Colon Cancer Histopathological Image Dataset (LC25000)
• Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniques
• Dynamic Channel Pruning: Feature Boosting and Suppression
• QuaterNet: A Quaternion-based Recurrent Model for Human Motion
• A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• Stereo Magnification: Learning View Synthesis using Multiplane Images
• A Discriminative Representation of Convolutional Features for Indoor Scene Recognition
• A new face database simultaneously acquired in visible, near infrared and thermal spectrum
• Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction
• Zipf's law in 50 languages: its structural pattern, linguistic interpretation, and cognitive motivation
• Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
• Light Field Compression with Disparity Guided Sparse Coding based on Structural Key Views
• Linear Mode Connectivity and the Lottery Ticket Hypothesis
• Convolution Neural Networks for diagnosing colon and lung cancer histopathological images
• Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective
• Video Object Segmentation with Re-identification
• BodyNet: Volumetric Inference of 3D Human Body Shapes
• Point-wise Map Recovery and Refinement from Functional Correspondence
• Self-Correction for Human Parsing
• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
• RVOS: End-to-End Recurrent Network for Video Object Segmentation
• Fast and Accurate Optical Flow based Depth Map Estimation from Light Fields
• Thin Structure Estimation with Curvature Regularization
• CamSwarm: Instantaneous Smartphone Camera Arrays for Collaborative Photography
• ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations
• Pruning Convolutional Neural Networks with Self-Supervision
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning
• Evaluating Contrastive Models for Instance-based Image Retrieval
• PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition
• Towards End-to-End Neural Face Authentication in the Wild -- Quantifying and Compensating for Directional Lighting Effects
• A Study of Efficient Light Field Subsampling and Reconstruction Strategies
• Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields
• Shape Analysis via Functional Map Construction and Bases Pursuit