Skeleton Transformer Networks: 3D Human Pose and Skinned Mesh from Single RGB Image

Abstract

In this paper, we present Skeleton Transformer Networks (SkeletonNet), an end-to-end framework that can predict not only 3D joint positions but also 3D angular pose (bone rotations) of a human skeleton from a single color image. This in turn allows us to generate skinned mesh animations. Here, we propose a two-step regression approach. The first step regresses bone rotations in order to obtain an initial solution by considering skeleton structure. The second step performs refinement based on heatmap regressor using a 3D pose representation called cross heatmap which stacks heatmaps of xy and zy coordinates. By training the network using the proposed 3D human pose dataset that is comprised of images annotated with 3D skeletal angular poses, we showed that SkeletonNet can predict a full 3D human pose (joint positions and bone rotations) from a single image in-the-wild.

arXiv commentACCV conference

References in corpus (8)

• Deep Residual Learning for Image Recognition
• DeepPose: Human Pose Estimation via Deep Neural Networks
• End-to-End Training of Deep Visuomotor Policies
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Multi-Context Attention for Human Pose Estimation
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Learning Feature Pyramids for Human Pose Estimation
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision

Cited by in corpus (1)

• Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representation