108 citations · 403 across the 25 of their papers we have counts for
25 papers
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer
Kunkun Pang, Dafei Qin, Yingruo Fan +4
Automatic gesture synthesis from speech is a topic that has attracted researchers for applications in remote communication, video games and Metaverse. Learning the mapping between…
Motion In-Betweening with Phase Manifolds
Paul Starke, Sebastian Starke, Taku Komura +1
This paper introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder. Our a…
Patch-Grid: An Efficient and Feature-Preserving Neural Implicit Surface Representation
Guying Lin, Lei Yang, Congyi Zhang +6
Neural implicit representations are widely used for 3D shape modeling due to their smoothness and compactness, but traditional MLP-based methods struggle with sharp features, such…
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
Yuan Liu, Peng Wang, Cheng Lin +5
We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.…
Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild
Dafei Qin, Jun Saito, Noam Aigerman +2
We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds…
F-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Peng Wang, Yuan Liu, Zhaoxi Chen +5
This paper presents a novel grid-based NeRF called F2-NeRF (Fast-Free-NeRF) for novel view synthesis, which enables arbitrary input camera trajectories and only costs a few minutes…
Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
Wenjia Wang, Yongtao Ge, Haiyi Mei +6
As it is hard to calibrate single-view RGB images in the wild, existing 3D human mesh reconstruction (3DHMR) methods either use a constant large focal length or estimate one based…
Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians
Jiawei Huang, Akito Iizuka, Hajime Tanaka +2
The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique. In this paper we propose a novel online framework to le…
NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies
Xiaoxiao Long, Cheng Lin, Lingjie Liu +5
We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. Recent advances in neural rendering based re…
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
Zhiyang Dou, Qingxuan Wu, Cheng Lin +5
In this paper, we introduce a set of simple yet effective TOken REduction (TORE) strategies for Transformer-based Human Mesh Recovery from monocular images. Current SOTA performanc…
Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos
Yilin Wen, Hao Pan, Lei Yang +3
Understanding dynamic hand motions and actions from egocentric RGB videos is a fundamental yet challenging task due to self-occlusion and ambiguity. To address occlusion and ambigu…
SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views
Xiaoxiao Long, Cheng Lin, Peng Wang +2
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images. This task becomes more difficult when only sparse imag…
Progressively-connected Light Field Network for Efficient View Synthesis
Peng Wang, Yuan Liu, Guying Lin +4
This paper presents a Progressively-connected Light Field network (ProLiF), for the novel view synthesis of complex forward-facing scenes. ProLiF encodes a 4D light field, which al…
Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions
Weilin Wan, Lei Yang, Lingjie Liu +7
Understanding human intentions during interactions has been a long-lasting theme, that has applications in human-robot interaction, virtual reality and surveillance. In this study,…
NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors
Jiepeng Wang, Peng Wang, Xiaoxiao Long +4
Reconstructing 3D indoor scenes from 2D images is an important task in many computer vision and graphics applications. A main challenge in this task is that large texture-less area…
Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
Yuan Liu, Yilin Wen, Sida Peng +4
In this paper, we present a generalizable model-free 6-DoF object pose estimator called Gen6D. Existing generalizable pose estimators either need high-quality object models or requ…
Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases
Ian Mason, Sebastian Starke, Taku Komura
Controlling the manner in which a character moves in a real-time animation system is a challenging task with useful applications. Existing style transfer systems require access to…
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation
Yingruo Fan, Zhaojiang Lin, Jun Saito +2
Speech-driven 3D facial animation with accurate lip synchronization has been widely studied. However, synthesizing realistic motions for the entire face during speech has rarely be…
FaceFormer: Speech-Driven 3D Facial Animation with Transformers
Yingruo Fan, Zhaojiang Lin, Jun Saito +2
Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data. Prior works typically focus on lea…
Coverage Axis: Inner Point Selection for 3D Shape Skeletonization
Zhiyang Dou, Cheng Lin, Rui Xu +4
In this paper, we present a simple yet effective formulation called Coverage Axis for 3D shape skeletonization. Inspired by the set cover problem, our key idea is to cover all the…
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
Peng Wang, Lingjie Liu, Yuan Liu +3
We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconst…
MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
Mingyi Shi, Kfir Aberman, Andreas Aristidou +4
We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video.While previous methods rely on either rigging or inve…
Technical Note: Generating Realistic Fighting Scenes by Game Tree
Hubert P. H. Shum, Taku Komura
Recently, there have been a lot of researches to synthesize / edit the motion of a single avatar in the virtual environment. However, there has not been so much work of simulating…
Learning natural locomotion behaviors for humanoid robots using human knowledge
Chuanyu Yang, Kai Yuan, Shuai Heng +2
This paper presents a new learning framework that leverages the knowledge from imitation learning, deep reinforcement learning, and control theories to achieve human-style locomoti…