380 citations · 994 across the 32 of their papers we have counts for
33 papers
Imagine yourself: Tuning-Free Personalized Image Generation
Zecheng He, Bo Sun, Felix Juefei-Xu +14
Diffusion models have demonstrated remarkable efficacy across various image-to-image tasks. In this research, we introduce Imagine yourself, a state-of-the-art model designed for p…
An Analysis on Quantizing Diffusion Transformers
Yuewei Yang, Jialiang Wang, Xiaoliang Dai +2
Diffusion Models (DMs) utilize an iterative denoising process to transform random noise into synthetic data. Initally proposed with a UNet structure, DMs excel at producing images…
Cache Me if You Can: Accelerating Diffusion Models through Block Caching
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld +11
Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion…
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
Feng Liang, Bichen Wu, Jialiang Wang +8
Diffusion models have transformed the image-to-image (I2I) synthesis and are now permeating into videos. However, the advancement of video-to-video (V2V) synthesis has been hampere…
Efficient Quantization Strategies for Latent Diffusion Models
Yuewei Yang, Xiaoliang Dai, Jialiang Wang +2
Latent Diffusion Models (LDMs) capture the dynamic evolution of latent variables over time, blending patterns and multimodality in a generative system. Despite the proficiency of L…
ControlRoom3D: Room Generation using Semantic Proxy Rooms
Jonas Schult, Sam Tsai, Lukas Höllein +11
Manually creating 3D environments for AR/VR applications is a complex process requiring expert knowledge in 3D modeling software. Pioneering works facilitate this process by genera…
Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
Xiaoliang Dai, Ji Hou, Chih-Yao Ma +23
Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text. However, these pre-trained models often face chal…
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Yonggan Fu, Yuecheng Li, Chenghui Li +4
Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence. However, there still exists one k…
DIME-FM: DIstilling Multimodal and Efficient Foundation Models
Ximeng Sun, Pengchuan Zhang, Peizhao Zhang +3
Large Vision-Language Foundation Models (VLFM), such as CLIP, ALIGN and Florence, are trained on large-scale datasets of image-caption pairs and achieve superior transferability an…
Pruning Compact ConvNets for Efficient Inference
Sayan Ghosh, Karthik Prasad, Xiaoliang Dai +4
Neural network pruning is frequently used to compress over-parameterized networks by large amounts, while incurring only marginal drops in generalization performance. However, the…
INGeo: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors
Chaojian Li, Bichen Wu, Albert Pumarola +3
We present a method that accelerates reconstruction of 3D scenes and objects, aiming to enable instant reconstruction on edge devices such as mobile phones and AR/VR headsets. Whil…
3D-Aware Encoding for Style-based Neural Radiance Fields
Yu-Jhe Li, Tao Xu, Bichen Wu +6
We tackle the task of NeRF inversion for style-based neural radiance fields, (e.g., StyleNeRF). In the task, we aim to learn an inversion function to project an input image to the…
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference
Haoran You, Yunyang Xiong, Xiaoliang Dai +5
Vision Transformers (ViTs) have shown impressive performance but still require a high computation cost as compared to convolutional neural networks (CNNs), one reason is that ViTs'…
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
Hyoukjun Kwon, Krishnakumar Nair, Jamin Seo +15
Real-time multi-task multi-model (MTMM) workloads, a new form of deep learning inference workloads, are emerging for applications areas like extended reality (XR) to support metave…
UmeTrack: Unified multi-view end-to-end hand tracking for VR
Shangchen Han, Po-chen Wu, Yubo Zhang +13
Real-time tracking of 3D hand pose in world space is a challenging problem and plays an important role in VR interaction. Existing work in this space are limited to either producin…
Token Merging: Your ViT But Faster
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai +3
We introduce Token Merging (ToMe), a simple method to increase the throughput of existing ViT models without needing to train. ToMe gradually combines similar tokens in a transform…
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
Feng Liang, Bichen Wu, Xiaoliang Dai +6
Open-vocabulary semantic segmentation aims to segment an image into semantic regions according to text descriptions, which may not have been seen during training. Recent two-stage…
Hydra Attention: Efficient Attention with Many Heads
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai +2
While transformers have begun to dominate many tasks in vision, applying them to large images is still computationally difficult. A large reason for this is that self-attention sca…
Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation
Bichen Wu, Ruizhe Cheng, Peizhao Zhang +3
Traditional computer vision models are trained to predict a fixed set of predefined categories. Recently, natural language has been shown to be a broader and richer source of super…
FBNetV5: Neural Architecture Search for Multiple Tasks in One Run
Bichen Wu, Chaojian Li, Hang Zhang +6
Neural Architecture Search (NAS) has been widely adopted to design accurate and efficient image classification models. However, applying NAS to a new computer vision task still req…
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation
Ruizhe Cheng, Bichen Wu, Peizhao Zhang +2
Traditional computer vision models are trained to predict a fixed set of predefined categories. Recently, natural language has been shown to be a broader and richer source of super…
Unbiased Teacher for Semi-Supervised Object Detection
Yen-Cheng Liu, Chih-Yao Ma, Zijian He +6
Semi-supervised learning, i.e., training networks with both labeled and unlabeled data, has made significant progress recently. However, existing works have primarily focused on im…
Low Bandwidth Video-Chat Compression using Deep Generative Models
Maxime Oquab, Pierre Stock, Oran Gafni +9
To unlock video chat for hundreds of millions of people hindered by poor connectivity or unaffordable data costs, we propose to authentically reconstruct faces on the receiver's de…
FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge
Bichen Wu, Qing He, Peizhao Zhang +3
Nowadays more and more applications can benefit from edge-based text-to-speech (TTS). However, most existing TTS models are too computationally expensive and are not flexible enoug…