32 citations · 137 across the 24 of their papers we have counts for
24 papers
Enhancing NeRF akin to Enhancing LLMs: Generalizable NeRF Transformer with Mixture-of-View-Experts
Wenyan Cong, Hanxue Liang, Peihao Wang +5
Cross-scene generalizable NeRF models, which can directly synthesize novel views of unseen scenes, have become a new spotlight of the NeRF field. Several existing attempts rely on…
INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing
Stefan Abi-Karam, Rishov Sarkar, Dejia Xu +3
An increasing number of researchers are finding use for nth-order gradient computations for a wide variety of applications, including graphics, meta-learning (MAML), scientific com…
Physics-Driven Turbulence Image Restoration with Stochastic Refinement
Ajay Jaiswal, Xingguang Zhang, Stanley H. Chan +1
Image distortion by atmospheric turbulence is a stochastic degradation, which is a critical problem in long-range optical imaging systems. A number of research has been conducted d…
Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap
Dejia Xu, Xingqian Xu, Wenyan Cong +2
Have you ever imagined how it would look if we placed new objects into paintings? For example, what would it look like if we placed a basketball into Claude Monet's ``Water Lilies,…
Polynomial Width is Sufficient for Set Representation with High-dimensional Features
Peihao Wang, Shenghao Yang, Shu Li +2
Set representation has become ubiquitous in deep learning for modeling the inductive bias of neural networks that are insensitive to the input order. DeepSets is the most widely us…
Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images
Panwang Pan, Zhiwen Fan, Brandon Y. Feng +3
The accurate estimation of six degrees-of-freedom (6DoF) object poses is essential for many applications in robotics and augmented reality. However, existing methods for 6DoF pose…
FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude
Feng Liu, Ryan Ashbaugh, Nicholas Chimitt +17
Whole-body biometric recognition is an important area of research due to its vast applications in law enforcement, border security, and surveillance. This paper presents the end-to…
HO: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Zhenyu Zhang, Ying Sheng, Tianyi Zhou +9
Large Language Models (LLMs), despite their recent impressive accomplishments, are notably cost-prohibitive to deploy, particularly for applications involving long-content generati…
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
Xingqian Xu, Jiayi Guo, Zhangyang Wang +3
Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches.…
POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference
Zhiwen Fan, Panwang Pan, Peihao Wang +4
Despite the significant progress in six degrees-of-freedom (6DoF) object pose estimation, existing methods have limited applicability in real-world scenarios involving embodied age…
MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition
Xinyu Gong, Sreyas Mohan, Naina Dhingra +4
In this paper, we study a novel problem in egocentric action recognition, which we term as "Multimodal Generalization" (MMG). MMG aims to study how systems can generalize when data…
Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts
Rishov Sarkar, Hanxue Liang, Zhiwen Fan +2
Computer vision researchers are embracing two promising paradigms: Vision Transformers (ViTs) and Multi-task Learning (MTL), which both show great performance but are computation-i…
In-Context Learning Unlocked for Diffusion Models
Zhendong Wang, Yifan Jiang, Yadong Lu +5
We present Prompt Diffusion, a framework for enabling in-context learning in diffusion-based generative models. Given a pair of task-specific example images, such as depth from/to…
Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation
Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal +4
For a complicated algorithm, its implementation by a human programmer usually starts with outlining a rough control flow followed by iterative enrichments, eventually yielding care…
AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection
Wentao Zhu, Yufang Huang, Xiufeng Xie +5
The short-form videos have explosive popularity and have dominated the new social media trends. Prevailing short-video platforms,~\textit{e.g.}, Kuaishou (Kwai), TikTok, Instagram…
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Zhendong Wang, Yifan Jiang, Huangjie Zheng +5
Diffusion models are powerful, but they require a lot of time and data to train. We propose Patch Diffusion, a generic patch-wise training framework, to significantly reduce the tr…
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan +4
Recent text-to-video generation approaches rely on computationally heavy training and require large-scale video datasets. In this paper, we introduce a new task of zero-shot text-t…
Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search
Wenqing Zheng, S P Sharan, Zhiwen Fan +3
Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural ne…
StegaNeRF: Embedding Invisible Information within Neural Radiance Fields
Chenxin Li, Brandon Y. Feng, Zhiwen Fan +2
Recent advances in neural rendering imply a future of widespread visual data distributions through sharing NeRF model weights. However, while common visual data (images and videos)…
AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training
Yifan Jiang, Peter Hedman, Ben Mildenhall +4
Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function. Though NeRF is able to render complex 3D scenes with view-dependent e…
NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views
Dejia Xu, Yifan Jiang, Peihao Wang +3
Virtual reality and augmented reality (XR) bring increasing demand for 3D content. However, creating high-quality 3D content requires tedious work that a human expert must do. In t…
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xingqian Xu, Zhangyang Wang, Eric Zhang +2
Recent advances in diffusion models have set an impressive milestone in many generation tasks, and trending works such as DALL-E2, Imagen, and Stable Diffusion have attracted great…
Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices
Yimeng Zhang, Akshay Karkal Kamath, Qiucheng Wu +6
In this paper, we propose a data-model-hardware tri-design framework for high-throughput, low-cost, and high-accuracy multi-object tracking (MOT) on High-Definition (HD) video stre…
NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes
Zhiwen Fan, Peihao Wang, Yifan Jiang +3
Neural volumetric representations have shown the potential that Multi-layer Perceptrons (MLPs) can be optimized with multi-view calibrated images to represent scene geometry and ap…