Publications (79)
The persistence of synchronization under -stable noise
Yanjie Zhang, Li Lin, Jinqiao Duan +1
This work is about the synchronization of nonlinear coupled dynamical systems driven by -stable noise. Firstly, we provide a novel technique to construct the relationship betwe…
GA-Sketching: Shape Modeling from Multi-View Sketching with Geometry-Aligned Deep Implicit Functions
Jie Zhou, Zhongjin Luo, Qian Yu +2
Sketch-based shape modeling aims to bridge the gap between 2D drawing and 3D modeling by providing an intuitive and accessible approach to create 3D shapes from 2D sketches. Howeve…
DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN
Wanchao Su, Hui Ye, Shu-Yu Chen +2
The research topic of sketch-to-portrait generation has witnessed a boost of progress with deep learning techniques. The recently proposed StyleGAN architectures achieve state-of-t…
NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks
Beijia Chen, Yuefan Shen, Hongbo Fu +3
In this paper, we present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. To achieve globally coherent reshap…
Sketch2Stress: Sketching with Structural Stress Awareness
Deng Yu, Chufeng Xiao, Manfred Lau +1
In the process of product design and digital fabrication, the structural analysis of a designed prototype is a fundamental and essential step. However, such a step is usually invis…
JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds
Zeyu Hu, Mingmin Zhen, Xuyang Bai +2
Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D sema…
Controllable Texture Tiling with Transformed RoPE-Enhanced Diffusion Models
Junrong Huang, Zhiyuan Zhang, Rui Tang +2
Realistic integration of user-specified textures into scene images is a fundamental task in computer graphics and image editing. While existing material transfer and reference-guid…
DeepPortraitDrawing: Generating Human Body Images from Freehand Sketches
Xian Wu, Chen Wang, Hongbo Fu +3
Researchers have explored various ways to generate realistic images from freehand sketches, e.g., for objects and human faces. However, how to generate realistic human body images…
SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
Deng Yu, Lei Li, Youyi Zheng +4
In this paper, we study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object and predict as ou…
Fast Sketch Segmentation and Labeling with Deep Learning
Lei Li, Hongbo Fu, Chiew-Lan Tai
We present a simple and efficient method based on deep learning to automatically decompose sketched objects into semantically valid parts. We train a deep neural network to transfe…
NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations
Keyu Wu, Yifan Ye, Lingchen Yang +3
Undoubtedly, high-fidelity 3D hair plays an indispensable role in digital humans. However, existing monocular hair modeling methods are either tricky to deploy in digital systems (…
GCRayDiffusion: Pose-Free Surface Reconstruction via Geometric Consistent Ray Diffusion
Li-Heng Chen, Zi-Xin Zou, Chang Liu +5
Accurate surface reconstruction from unposed images is crucial for efficient 3D object or scene creation. However, it remains challenging, particularly for the joint camera pose es…
SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling
Zhongjin Luo, Dong Du, Heming Zhu +3
Modeling 3D avatars benefits various application scenarios such as AR/VR, gaming, and filming. Character faces contribute significant diversity and vividity as a vital component of…
VC-Agent: An Interactive Agent for Customized Video Dataset Collection
Yidan Zhang, Mutian Xu, Yiming Hao +6
Facing scaling laws, video data from the internet becomes increasingly important. However, collecting extensive videos that meet specific needs is extremely labor-intensive and tim…
Real-time 3D-aware Portrait Video Relighting
Ziqi Cai, Kaiwen Jiang, Shu-Yu Chen +4
Synthesizing realistic videos of talking faces under custom lighting conditions and viewing angles benefits various downstream applications like video conferencing. However, most e…
LPFF: A Portrait Dataset for Face Generators Across Large Poses
Yiqian Wu, Jing Zhang, Hongbo Fu +1
The creation of 2D realistic facial images and 3D face shapes using generative networks has been a hot topic in recent years. Existing face generators exhibit exceptional performan…
Mesh-based Gaussian Splatting for Real-time Large-scale Deformation
Lin Gao, Jie Yang, Bo-Tao Zhang +4
Neural implicit representations, including Neural Distance Fields and Neural Radiance Fields, have demonstrated significant capabilities for reconstructing surfaces with complicate…
D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features
Xuyang Bai, Zixin Luo, Lei Zhou +3
A successful point cloud registration often lies on robust establishment of sparse matches through discriminative 3D local features. Despite the fast evolution of learning-based 3D…
SDM-NET: Deep Generative Network for Structured Deformable Mesh
Lin Gao, Jie Yang, Tong Wu +4
We introduce SDM-NET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed…
Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling
Gongye Liu, Bo Yang, Yida Zhi +8
Preference optimization for diffusion and flow-matching models relies on reward functions that are both discriminatively robust and computationally efficient. Vision-Language Model…
GCN-Denoiser: Mesh Denoising with Graph Convolutional Networks
Yuefan Shen, Hongbo Fu, Zhongshuo Du +5
In this paper, we present GCN-Denoiser, a novel feature-preserving mesh denoising method based on graph convolutional networks (GCNs). Unlike previous learning-based mesh denoising…
Autocomplete Repetitive Stroking with Image Guidance
Yilan Chen, Kin Chung Kwan, Li-Yi Wei +1
Image-guided drawing can compensate for the lack of skills but often requires a significant number of repetitive strokes to create textures. Existing automatic stroke synthesis met…
PartHOI: Part-based Hand-Object Interaction Transfer via Generalized Cylinders
Qiaochu Wang, Chufeng Xiao, Manfred Lau +1
Learning-based methods to understand and model hand-object interactions (HOI) require a large amount of high-quality HOI data. One way to create HOI data is to transfer hand poses…
Weak order in averaging principle for two-time-scale stochastic partial differential equations
Hongbo Fu, Li Wan, Jicheng Liu +1
This work is devoted to averaging principle of a two-time-scale stochastic partial differential equation on a bounded interval , where both the fast and slow components are…
SketchVideo: Sketch-based Video Generation and Editing
Feng-Lin Liu, Hongbo Fu, Xintao Wang +4
Video generation and editing conditioned on text prompts or images have undergone significant advancements. However, challenges remain in accurately controlling global layout and g…
Region-Aware Color Smudging
Ying Jiang, Pengfei Xu, Congyi Zhang +3
Color smudge operations from digital painting software enable users to create natural shading effects in high-fidelity paintings by interactively mixing colors. To precisely contro…
Human Motion Transfer with 3D Constraints and Detail Enhancement
Yang-Tian Sun, Qian-Cheng Fu, Yue-Ren Jiang +4
We propose a new method for realistic human motion transfer using a generative adversarial network (GAN), which generates a motion video of a target character imitating actions of…
VistaGEN: Consistent Driving Video Generation with Fine-Grained Control Using Multiview Visual-Language Reasoning
Li-Heng Chen, Ke Cheng, Yahui Liu +3
Driving video generation has achieved much progress in controllability, video resolution, and length, but fails to support fine-grained object-level controllability for diverse dri…
MonoHair: High-Fidelity Hair Modeling from a Monocular Video
Keyu Wu, Lingchen Yang, Zhiyi Kuang +6
Undoubtedly, high-fidelity 3D hair is crucial for achieving realism, artistic expression, and immersion in computer graphics. While existing 3D hair modeling methods have achieved…
Weak order in averaging principle for stochastic differential equations with jumps
Bengong Zhang, Hongbo Fu, Li Wan +1
The present article deals with the averaging principle for a two-time-scale system of jump-diffusion stochastic differential equation. Under suitable conditions, the weak error is…
MoGraphGPT: Creating Interactive Scenes Using Modular LLM and Graphical Control
Hui Ye, Chufeng Xiao, Jiaye Leng +2
Creating interactive scenes often involves complex programming tasks. Although large language models (LLMs) like ChatGPT can generate code from natural language, their output is of…
DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering
Xin Yang, Wenbo Hu, Dawei Wang +5
In this paper, we present DEMC, a deep Dual-Encoder network to remove Monte Carlo noise efficiently while preserving details. Denoising Monte Carlo rendering is different from natu…
SketchHairSalon: Deep Sketch-based Hair Image Synthesis
Chufeng Xiao, Deng Yu, Xiaoguang Han +2
Recent deep generative models allow real-time generation of hair images from sketch inputs. Existing solutions often require a user-provided binary mask to specify a target hair sh…
SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design
Zhongjin Luo, Jie Zhou, Heming Zhu +3
Head shapes play an important role in 3D character design. In this work, we propose SimpModeling, a novel sketch-based system for helping users, especially amateur users, easily mo…
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control
Shu-Yu Chen, Feng-Lin Liu, Yu-Kun Lai +4
Recent facial image synthesis methods have been mainly based on conditional generative models. Sketch-based conditions can effectively describe the geometry of faces, including the…
LUCSS: Language-based User-customized Colourization of Scene Sketches
Changqing Zou, Haoran Mo, Ruofei Du +3
We introduce LUCSS, a language-based system for interactive col- orization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks traine…
Sketch3DVE: Sketch-based 3D-Aware Scene Video Editing
Feng-Lin Liu, Shi-Yang Li, Yan-Pei Cao +2
Recent video editing methods achieve attractive results in style transfer or appearance modification. However, editing the structural content of 3D scenes in videos remains challen…
On the limit distribution for stochastic differential equations driven by cylindrical non-symmetric -stable Lévy processes
Ting Li, Hongbo Fu, Xianming Liu
This article deals with the limit distribution for a stochastic differential equation driven by a non-symmetric cylindrical -stable process. Under suitable conditions, it is pr…
Deep Generation of Face Images from Sketches
Shu-Yu Chen, Wanchao Su, Lin Gao +2
Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiri…
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing
Chufeng Xiao, Hongbo Fu
Personalization techniques for large text-to-image (T2I) models allow users to incorporate new concepts from reference images. However, existing methods primarily rely on textual d…
WristSketcher: Creating Dynamic Sketches in AR with a Sensing Wristband
Enting Ying, Tianyang Xiong, Shihui Guo +3
Restricted by the limited interaction area of native AR glasses (e.g., touch bars), it is challenging to create sketches in AR glasses. Recent works have attempted to use mobile de…
Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control
Boyu Li, Linjie Qiu, Lin-Ping Yuan +4
Controlling attributes is a critical step toward achieving the final creative outcome, yet current approaches fall short in supporting users in the iterative refinement of generati…
From Rigging to Waving: 3D-Guided Diffusion for Natural Animation of Hand-Drawn Characters
Jie Zhou, Linzi Qu, Miu-Ling Lam +1
Hand-drawn character animation is a vibrant field in computer graphics, presenting challenges in achieving geometric consistency while conveying expressive motion. Traditional skel…
Color Orchestra: Ordering Color Palettes for Interpolation and Prediction
Huy Q. Phan, Hongbo Fu, Antoni B. Chan
Color theme or color palette can deeply influence the quality and the feeling of a photograph or a graphical design. Although color palettes may come from different sources such as…
3DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses
Yiqian Wu, Hao Xu, Xiangjun Tang +3
3D-aware face generators are typically trained on 2D real-life face image datasets that primarily consist of near-frontal face data, and as such, they are unable to construct one-q…
DeepSketchHair: Deep Sketch-based 3D Hair Modeling
Yuefan Shen, Changgeng Zhang, Hongbo Fu +2
We present sketchhair, a deep learning based tool for interactive modeling of 3D hair from 2D sketches. Given a 3D bust model as reference, our sketching system takes as input a us…
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Xuyang Bai, Zeyu Hu, Xinge Zhu +4
LiDAR and camera are two important sensors for 3D object detection in autonomous driving. Despite the increasing popularity of sensor fusion in this field, the robustness against i…
WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration
Lei Li, Hongbo Fu, Maks Ovsjanikov
In this work, we present a novel method called WSDesc to learn 3D local descriptors in a weakly supervised manner for robust point cloud registration. Our work builds upon recent 3…
End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds
Lei Li, Siyu Zhu, Hongbo Fu +2
In this work, we propose an end-to-end framework to learn local multi-view descriptors for 3D point clouds. To adopt a similar multi-view representation, existing studies use hand-…
The impact of multiplicative noise in SPDEs close to bifurcation via amplitude equations
Hongbo Fu, Dirk Blömker
This article deals with the approximation of a stochastic partial differential equation (SPDE) via amplitude equations. We consider an SPDE with a cubic nonlinearity perturbed by a…
NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields
Kaiwen Jiang, Shu-Yu Chen, Feng-Lin Liu +2
Recent methods for synthesizing 3D-aware face images have achieved rapid development thanks to neural radiance fields, allowing for high quality and fast inference speed. However,…
ROAR-3D: Routing Arbitrary Views for High-Fidelity 3D Generation
Hanxiao Sun, Mingxin Yang, Shuhui Yang +5
Single-image-to-3D generative models can now produce high-quality geometry, yet conditioning on a single view inevitably introduces ambiguity about unseen regions. Multi-view condi…
StructLayoutFormer:Conditional Structured Layout Generation via Structure Serialization and Disentanglement
Xin Hu, Pengfei Xu, Jin Zhou +2
Structured layouts are preferable in many 2D visual contents (\eg, GUIs, webpages) since the structural information allows convenient layout editing. Computational frameworks can h…
Sketch2Human: Deep Human Generation with Disentangled Geometry and Appearance Control
Linzi Qu, Jiaxiang Shang, Hui Ye +2
Geometry- and appearance-controlled full-body human image generation is an interesting but challenging task. Existing solutions are either unconditional or dependent on coarse cond…
PoseCoach: A Customizable Analysis and Visualization System for Video-based Running Coaching
Jingyuan Liu, Nazmus Saquib, Zhutian Chen +4
Videos are an accessible form of media for analyzing sports postures and providing feedback to athletes. Existing sport-specific systems embed bespoke human pose attributes and thu…
Slow Manifolds for Multi-Time-Scale Stochastic Evolutionary Systems
Hongbo Fu, Xianming Liu, Jinqiao Duan
This article deals with invariant manifolds for infinite dimensional random dynamical systems with different time scales. Such a random system is generated by a coupled system of f…
DrawingSpinUp: 3D Animation from Single Character Drawings
Jie Zhou, Chufeng Xiao, Miu-Ling Lam +1
Animating various character drawings is an engaging visual content creation task. Given a single character drawing, existing animation methods are limited to flat 2D motions and th…
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency
Xuyang Bai, Zixin Luo, Lei Zhou +5
Removing outlier correspondences is one of the critical steps for successful feature-based point cloud registration. Despite the increasing popularity of introducing deep learning…
LiDAL: Inter-frame Uncertainty Based Active Learning for 3D LiDAR Semantic Segmentation
Zeyu Hu, Xuyang Bai, Runze Zhang +4
We propose LiDAL, a novel active learning method for 3D LiDAR semantic segmentation by exploiting inter-frame uncertainty among LiDAR frames. Our core idea is that a well-trained m…
Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition
Lei Li, Changqing Zou, Youyi Zheng +3
Freehand sketching is a dynamic process where points are sequentially sampled and grouped as strokes for sketch acquisition on electronic devices. To recognize a sketched object, m…
Weak order in averaging principle for stochastic wave equations with a fast oscillation
Hongbo Fu, Li Wan, Jicheng Liu +1
This article deals with the weak errors for averaging principle for a stochastic wave equation in a bounded interval , perturbed by a oscillating term arising as the solutio…
VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation
Zeyu Hu, Xuyang Bai, Jiaxiang Shang +6
In recent years, sparse voxel-based methods have become the state-of-the-arts for 3D semantic segmentation of indoor scenes, thanks to the powerful 3D CNNs. Nevertheless, being obl…
SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks
Lumin Yang, Jiajie Zhuang, Hongbo Fu +3
We introduce SketchGNN, a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph, w…
StyleRetoucher: Generalized Portrait Image Retouching with GAN Priors
Wanchao Su, Can Wang, Chen Liu +3
Creating fine-retouched portrait images is tedious and time-consuming even for professional artists. There exist automatic retouching methods, but they either suffer from over-smoo…
MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors
Zi-Xin Zou, Shi-Sheng Huang, Yan-Pei Cao +3
High-fidelity 3D scene reconstruction from monocular videos continues to be challenging, especially for complete and fine-grained geometry reconstruction. The previous 3D reconstru…
SketchDynamics: Exploring Free-Form Sketches for Dynamic Intent Expression in Animation Generation
Boyu Li, Lin-Ping Yuan, Zeyu Wang +1
Sketching provides an intuitive way to convey dynamic intent in animation authoring (i.e., how elements change over time and space), making it a natural medium for automatic conten…
Active Object Reconstruction Using a Guided View Planner
Xin Yang, Yuanbo Wang, Yaru Wang +4
Inspired by the recent advance of image-based object reconstruction using deep learning, we present an active reconstruction model using a guided view planner. We aim to reconstruc…
DisPad: Flexible On-Body Displacement of Fabric Sensors for Robust Joint-Motion Tracking
Xiaowei Chen, Xiao Jiang, Jiawei Fang +5
The last few decades have witnessed an emerging trend of wearable soft sensors; however, there are important signal-processing challenges for soft sensors that still limit their pr…
Mesh Variational Autoencoders with Edge Contraction Pooling
Yu-Jie Yuan, Yu-Kun Lai, Jie Yang +2
3D shape analysis is an important research topic in computer vision and graphics. While existing methods have generalized image-based deep learning to meshes using graph-based conv…
EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation
Songlin Yang, Haobin Zhong, Ruilin Zhang +23
The rapid evolution of generative video foundation models has propelled the field toward professional-grade cinematic synthesis. To achieve such demanding quality, the community tr…
Effective dynamicsof a coupled microscopic-macroscopic stochastic system
Jian Ren, Hongbo Fu, Daomin Cao +1
A conceptual model for microscopic-macroscopic slow-fast stochastic systems is considered. A dynamical reduction procedure is presented in order to extract effective dynamics for t…
Strong convergence rate in averaging principle for stochastic hyperbolic-parabolic equations with two time-scales
Hongbo Fu, Li Wan, Jicheng Liu +1
In this article, we investigate averaging principle for stochastic hyperbolic-parabolic equations with two time-scales, in which both the slow and fast components are perturbed by…
Real-and-Present: Investigating the Use of Life-Size 2D Video Avatars in HMD-Based AR Teleconferencing
Xuanyu Wang, Weizhan Zhang, Christian Sandor +1
Augmented Reality (AR) teleconferencing allows separately located users to interact with each other in 3D through agents in their own physical environments. Existing methods levera…
DifferSketching: How Differently Do People Sketch 3D Objects?
Chufeng Xiao, Wanchao Su, Jing Liao +3
Multiple sketch datasets have been proposed to understand how people draw 3D objects. However, such datasets are often of small scale and cover a small set of objects or categories…
Temporal Upsampling of Depth Maps Using a Hybrid Camera
Ming-Ze Yuan, Lin Gao, Hongbo Fu +1
In recent years, consumer-level depth cameras have been adopted for various applications. However, they often produce depth maps at only a moderately high frame rate (approximately…
Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made Objects
Deng Yu, Manfred Lau, Lin Gao +1
We present a novel freehand sketch beautification method, which takes as input a freely drawn sketch of a man-made object and automatically beautifies it both geometrically and str…
Scale-aware Insertion of Virtual Objects in Monocular Videos
Songhai Zhang, Xiangli Li, Yingtian Liu +1
In this paper, we propose a scale-aware method for inserting virtual objects with proper sizes into monocular videos. To tackle the scale ambiguity problem of geometry recovery fro…
SketchDream: Sketch-based Text-to-3D Generation and Editing
Feng-Lin Liu, Hongbo Fu, Yu-Kun Lai +1
Existing text-based 3D generation methods generate attractive results but lack detailed geometry control. Sketches, known for their conciseness and expressiveness, have contributed…
Sketch-based Shape Retrieval using Pyramid-of-Parts
Changqing Zou, Zhe Huang, Rynson W. H. Lau +2
We present a multi-scale approach to sketch-based shape retrieval. It is based on a novel multi-scale shape descriptor called Pyramidof- Parts, which encodes the features and spati…