Publications (42)
WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds
Peizhuo Li, Sebastian Starke, Yuting Ye +1
We present a new approach for understanding the periodicity structure and semantics of motion datasets, independently of the morphology and skeletal structure of characters. Unlike…
QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
Alexander Winkler, Jungdam Won, Yuting Ye
Real-time tracking of human body motion is crucial for interactive and immersive experiences in AR/VR. However, very limited sensor data about the body is available from standalone…
MOCHA: Real-Time Motion Characterization via Context Matching
Deok-Kyeong Jang, Yuting Ye, Jungdam Won +1
Transforming neutral, characterless input motions to embody the distinct style of a notable character in real time is highly compelling for character animation. This paper introduc…
Exploiting ITO colloidal nanocrystals for ultrafast pulse generation
Qiangbing Guo, Yudong Cui, Yunhua Yao +7
Dynamical materials that capable of responding to optical stimuli have always been pursued for designing novel photonic devices and functionalities, of which the response speed and…
QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors
Sunmin Lee, Sebastian Starke, Yuting Ye +2
Replicating a user's pose from only wearable sensors is important for many AR/VR applications. Most existing methods for motion tracking avoid environment interaction apart from fo…
Ranking hierarchical multi-label classification results with mLPRs
Yuting Ye, Christine Ho, Ci-Ren Jiang +2
Hierarchical multi-label classification (HMC) has gained considerable attention in recent decades. A seminal line of HMC research addresses the problem in two stages: first, traini…
Paired Test of Matrix Graphs and Brain Connectivity Analysis
Yuting Ye, Yin Xia, Lexin Li
Inferring brain connectivity network and quantifying the significance of interactions between brain regions are of paramount importance in neuroscience. Although there have recentl…
Learning Warped Guidance for Blind Face Restoration
Xiaoming Li, Ming Liu, Yuting Ye +3
This paper studies the problem of blind face restoration from an unconstrained blurry, noisy, low-resolution, or compressed image (i.e., degraded observation). For better recovery…
HONES: A Fast and Tuning-free Homotopy Method For Online Newton Step
Yuting Ye, Lihua Lei, Cheng Ju
In this article, we develop and analyze a homotopy continuation method, referred to as HONES , for solving the sequential generalized projections in Online Newton Step, as well as…
Rethinking the Data Annotation Process for Multi-view 3D Pose Estimation with Active Learning and Self-Training
Qi Feng, Kun He, He Wen +2
Pose estimation of the human body and hands is a fundamental problem in computer vision, and learning-based solutions require a large amount of annotated data. In this work, we imp…
EgoLM: Multi-Modal Language Model of Egocentric Motions
Fangzhou Hong, Vladimir Guzov, Hyo Jin Kim +4
As the prevalence of wearable devices, learning egocentric motions becomes essential to develop contextual AI. In this work, we present EgoLM, a versatile framework that tracks and…
Graphene/silicon heterojunction for reconfigurable phase-relevant activation function in coherent optical neural networks
Chuyu Zhong, Kun Liao, Tianxiang Dai +18
Optical neural networks (ONNs) herald a new era in information and communication technologies and have implemented various intelligent applications. In an ONN, the activation funct…
Dancing Points: Synthesizing Ballroom Dancing with Three-Point Inputs
Peizhuo Li, Sebastian Starke, Yuting Ye +1
Ballroom dancing is a structured yet expressive motion category. Its highly diverse movement and complex interactions between leader and follower dancers make the understanding and…
DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics
Yifeng Jiang, Jungdam Won, Yuting Ye +1
Synthesizing realistic human movements, dynamically responsive to the environment, is a long-standing objective in character animation, with applications in computer vision, sports…
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures
Abhronil Sengupta, Yuting Ye, Robert Wang +2
Over the past few years, Spiking Neural Networks (SNNs) have become popular as a possible pathway to enable low-power event-driven neuromorphic hardware. However, their application…
Physics-based Motion Retargeting from Sparse Inputs
Daniele Reda, Jungdam Won, Yuting Ye +2
Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/…
Identity-Disentangled Neural Deformation Model for Dynamic Meshes
Binbin Xu, Lingni Ma, Yuting Ye +3
Neural shape models can represent complex 3D shapes with a compact latent space. When applied to dynamically deforming shapes such as the human hands, however, they would need to p…
The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models
Wenpin Tang, Yuting Ye
Motivated by recent works on the high-dimensional logistic regression, we establish that the existence of the maximum likelihood estimate exhibits a phase transition for a wide ran…
HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Device
Vladimir Guzov, Yifeng Jiang, Fangzhou Hong +5
This paper investigates the generation of realistic full-body human motion using a single head-mounted device with an outward-facing color camera and the ability to perform visual…
GIRAF: Towards Generalizable Human Interactions with Articulated Objects
Xiaohan Zhang, Sebastian Starke, Alexander Winkler +3
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and vir…
CHOICE: Coordinated Human-Object Interaction in Cluttered Environments for Pick-and-Place Actions
Jintao Lu, He Zhang, Yuting Ye +3
Animating human-scene interactions such as pick-and-place tasks in cluttered, complex layouts is a challenging task, with objects of a wide variation of geometries and articulation…
Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculum
Yunbo Zhang, Alexander Clegg, Sehoon Ha +2
In-hand object manipulation is challenging to simulate due to complex contact dynamics, non-repetitive finger gaits, and the need to indirectly control unactuated objects. Further…
AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation
Thu Nguyen-Phuoc, Gabriel Schwartz, Yuting Ye +2
This paper presents a method that can quickly adapt dynamic 3D avatars to arbitrary text descriptions of novel styles. Among existing approaches for avatar stylization, direct opti…
Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation
Yifeng Jiang, Yuting Ye, Deepak Gopinath +3
Real-time human motion reconstruction from a sparse set of (e.g. six) wearable IMUs provides a non-intrusive and economic approach to motion capture. Without the ability to acquire…
Understanding the role of importance weighting for deep learning
Da Xu, Yuting Ye, Chuanwei Ruan
The recent paper by Byrd & Lipton (2019), based on empirical observations, raises a major concern on the impact of importance weighting for the over-parameterized deep learning mod…
Disentangling Pose from Appearance in Monochrome Hand Images
Yikang Li, Chris Twigg, Yuting Ye +2
Hand pose estimation from the monocular 2D image is challenging due to the variation in lighting, appearance, and background. While some success has been achieved using deep neural…
Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild
Lingni Ma, Yuting Ye, Fangzhou Hong +14
We introduce Nymeria - a large-scale, diverse, richly annotated human motion dataset collected in the wild with multiple multimodal egocentric devices. The dataset comes with a) fu…
MHR: Momentum Human Rig
Aaron Ferguson, Ahmed A. A. Osman, Berta Bescos +44
We present MHR, a parametric human body model that combines the decoupled skeleton/shape paradigm of ATLAS with a flexible, modern rig and pose corrective system inspired by the Mo…
ChildlikeSHAPES: Semantic Hierarchical Region Parsing for Animating Figure Drawings
Astitva Srivastava, Harrison Jesse Smith, Thu Nguyen-Phuoc +1
Childlike human figure drawings represent one of humanity's most accessible forms of character expression, yet automatically analyzing their contents remains a significant challeng…
Sonication-assisted synthesis of semiconducting atomic crystals of germanium monoselenide: An isostructural and isoelectronic analog of black phosphorous
Yuting Ye, Qiangbing Guo, Xiaofeng Liu +4
Monochalcogenides of germanium (or tin) are considered as stable isoelectronic and isostructural analogue of black phosphorous. Their two-dimensional (2D) forms have been just pred…
Bayes EMbedding (BEM): Refining Representation by Integrating Knowledge Graphs and Behavior-specific Networks
Yuting Ye, Xuwu Wang, Jiangchao Yao +4
Low-dimensional embeddings of knowledge graphs and behavior graphs have proved remarkably powerful in varieties of tasks, from predicting unobserved edges between entities to conte…
On-Chip Vectorial Structured Light Manipulation via Inverse Design
Xiaobin Lin, Maoliang Wei, Kunhao Lei +12
On-chip structured light, with potentially infinite complexity, has emerged as a linchpin in the realm of integrated photonics. However, the realization of arbitrarily tailoring a…
NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data
Daniel DeTone, Federica Bogo, Eric-Tuan Le +7
The Nymeria Dataset, released in 2024, is a large-scale collection of in-the-wild human activities captured with multiple egocentric wearable devices that are spatially localized a…
Simulation and Retargeting of Complex Multi-Character Interactions
Yunbo Zhang, Deepak Gopinath, Yuting Ye +3
We present a method for reproducing complex multi-character interactions for physically simulated humanoid characters using deep reinforcement learning. Our method learns control p…
HierLPR: Decision making in hierarchical multi-label classification with local precision rates
Christine Ho, Yuting Ye, Ci-Ren Jiang +2
In this article we propose a novel ranking algorithm, referred to as HierLPR, for the multi-label classification problem when the candidate labels follow a known hierarchical struc…
Animating Childlike Drawings with 2.5D Character Rigs
Harrison Jesse Smith, Nicky He, Yuting Ye
Drawing is a fun and intuitive way to create a character, accessible even to small children. However, animating 2D figure drawings is a much more challenging task, requiring specia…
From Intervention to Domain Transportation: A Novel Perspective to Optimize Recommendation
Da Xu, Yuting Ye, Chuanwei Ruan
The interventional nature of recommendation has attracted increasing attention in recent years. It particularly motivates researchers to formulate learning and evaluating recommend…
Towards Robust Off-policy Learning for Runtime Uncertainty
Da Xu, Yuting Ye, Chuanwei Ruan +1
Off-policy learning plays a pivotal role in optimizing and evaluating policies prior to the online deployment. However, during the real-time serving, we observe varieties of interv…
Binomial Mixture Model With U-shape Constraint
Yuting Ye, Peter J. Bickel
In this article, we study the binomial mixture model under the regime that the binomial size can be relatively large compared to the sample size . This project is motivated…
Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels
Yi Zhou, Chenglei Wu, Zimo Li +5
Learning latent representations of registered meshes is useful for many 3D tasks. Techniques have recently shifted to neural mesh autoencoders. Although they demonstrate higher pre…
DivaTrack: Diverse Bodies and Motions from Acceleration-Enhanced Three-Point Trackers
Dongseok Yang, Jiho Kang, Lingni Ma +3
Full-body avatar presence is crucial for immersive social and environmental interactions in digital reality. However, current devices only provide three six degrees of freedom (DOF…
From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Models
German Barquero, Nadine Bertsch, Manojkumar Marramreddy +8
In extended reality (XR), generating full-body motion of the users is important to understand their actions, drive their virtual avatars for social interaction, and convey a realis…