36 citations · 167 across the 25 of their papers we have counts for
25 papers
Movie Gen: A Cast of Media Foundation Models
Adam Polyak, Amit Zohar, Andrew Brown +85
We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabili…
Imagine yourself: Tuning-Free Personalized Image Generation
Zecheng He, Bo Sun, Felix Juefei-Xu +14
Diffusion models have demonstrated remarkable efficacy across various image-to-image tasks. In this research, we introduce Imagine yourself, a state-of-the-art model designed for p…
MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers
Haoyu Ma, Shahin Mahdizadehaghdam, Bichen Wu +5
Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly…
Instance Tracking in 3D Scenes from Egocentric Videos
Yunhan Zhao, Haoyu Ma, Shu Kong +1
Egocentric sensors such as AR/VR devices capture human-object interactions and offer the potential to provide task-assistance by recalling 3D locations of objects of interest in th…
CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer
Haoyu Ma, Tong Zhang, Shanlin Sun +3
Reconstructing personalized animatable head avatars has significant implications in the fields of AR/VR. Existing methods for achieving explicit face control of 3D Morphable Models…
Light Field Diffusion for Single-View Novel View Synthesis
Yifeng Xiong, Haoyu Ma, Shanlin Sun +3
Single-view novel view synthesis (NVS), the task of generating images from new viewpoints based on a single reference image, is important but challenging in computer vision. Recent…
Hybrid-CSR: Coupling Explicit and Implicit Shape Representation for Cortical Surface Reconstruction
Shanlin Sun, Thanh-Tung Le, Chenyu You +6
We present Hybrid-CSR, a geometric deep-learning model that combines explicit and implicit shape representations for cortical surface reconstruction. Specifically, Hybrid-CSR begin…
Localized Region Contrast for Enhancing Self-Supervised Learning in Medical Image Segmentation
Xiangyi Yan, Junayed Naushad, Chenyu You +6
Recent advancements in self-supervised learning have demonstrated that effective visual representations can be learned from unlabeled images. This has led to increased interest in…
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Zhenglun Kong, Haoyu Ma, Geng Yuan +12
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit the…
Identity-Aware Hand Mesh Estimation and Personalization from RGB Images
Deying Kong, Linguang Zhang, Liangjian Chen +6
Reconstructing 3D hand meshes from monocular RGB images has attracted increasing amount of attention due to its enormous potential applications in the field of AR/VR. Most state-of…
PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation
Haoyu Ma, Zhe Wang, Yifei Chen +6
Recently, the vision transformer and its variants have played an increasingly important role in both monocular and multi-view human pose estimation. Considering image patches as to…
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
Zhengang Li, Mengshu Sun, Alec Lu +9
Vision transformers (ViTs) are emerging with significantly improved accuracy in computer vision tasks. However, their complex architecture and enormous computation/storage demand i…
Training Your Sparse Neural Network Better with Any Mask
Ajay Jaiswal, Haoyu Ma, Tianlong Chen +2
Pruning large neural networks to create high-quality, independently trainable sparse masks, which can maintain similar performance to their dense counterparts, is very desirable du…
Medical Image Registration via Neural Fields
Shanlin Sun, Kun Han, Chenyu You +8
Image registration is an essential step in many medical image analysis tasks. Traditional methods for image registration are primarily optimization-driven, finding the optimal defo…
Sparsity Winning Twice: Better Robust Generalization from More Efficient Training
Tianlong Chen, Zhenyu Zhang, Pengjun Wang +4
Recent studies demonstrate that deep networks, even robustified by the state-of-the-art adversarial training (AT), still suffer from large robust generalization gaps, in addition t…
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
Mengshu Sun, Haoyu Ma, Guoliang Kang +5
The transformer architectures with attention mechanisms have obtained success in Nature Language Processing (NLP), and Vision Transformers (ViTs) have recently extended the applica…
Diffeomorphic Image Registration with Neural Velocity Field
Kun Han, Shanlin sun, Xiangyi Yan +6
Diffeomorphic image registration, offering smooth transformation and topology preservation, is required in many medical image analysis tasks.Traditional methods impose certain mode…
AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation
Xiangyi Yan, Hao Tang, Shanlin Sun +3
Recent advances in transformer-based models have drawn attention to exploring these techniques in medical image segmentation, especially in conjunction with the U-Net model (or its…
TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation
Haoyu Ma, Liangjian Chen, Deying Kong +7
Estimating the 2D human poses in each view is typically the first step in calibrated multi-view 3D pose estimation. But the performance of 2D pose detectors suffers from challengin…
SGE net: Video object detection with squeezed GRU and information entropy map
Rui Su, Wenjing Huang, Haoyu Ma +2
Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challeng…
Undistillable: Making A Nasty Teacher That CANNOT teach students
Haoyu Ma, Tianlong Chen, Ting-Kuei Hu +3
Knowledge Distillation (KD) is a widely used technique to transfer knowledge from pre-trained teacher models to (usually more lightweight) student models. However, in certain situa…
SIA-GCN: A Spatial Information Aware Graph Neural Network with 2D Convolutions for Hand Pose Estimation
Deying Kong, Haoyu Ma, Xiaohui Xie
Graph Neural Networks (GNNs) generalize neural networks from applications on regular structures to applications on arbitrary graphs, and have shown success in many application doma…
Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation
Deying Kong, Haoyu Ma, Yifei Chen +1
In this paper, we propose a new architecture named Rotation-invariant Mixed Graphical Model Network (R-MGMN) to solve the problem of 2D hand pose estimation from a monocular RGB im…
Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation
Yifei Chen, Haoyu Ma, Deying Kong +4
Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand. Current approaches often rely on a pop…