1.3k citations · 1.5k across the 47 of their papers we have counts for
36 papers · 1 filter
Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyond
Simone Facchiano, Jan Eric Lenssen, Bernt Schiele +3
As state-of-the-art text-to-image flow models achieve near-photorealistic quality, controlling their outputs, e.g., suppressing harmful content while promoting benign alternatives,…
Dynamic Inverse Rendering for Enhanced Material-Lighting Decomposition
Raza Yunus, Benjamin Ummenhofer, Jan Eric Lenssen +1
Decomposing outgoing surface radiance into material and illumination during inverse rendering is essential for applications such as relighting and augmented reality, yet it is seve…
CSFlow: Aligning Flow Matching with Human Contrast Sensitivity
Malgorzata Galinska, Bart Pogodzinski, Jan Eric Lenssen
We introduce Contrast Sensitive Flow (CSFlow), a weighting scheme that connects the human eye's Contrast Sensitivity Function (CSF) to the iterative denoising steps of flow matchin…
MixFlow: Mixed Source Distributions Improve Rectified Flows
Nazir Nayal, Christopher Wewer, Jan Eric Lenssen
Diffusion models and their variations, such as rectified flows, generate diverse and high-quality images, but they are still hindered by slow iterative sampling caused by the highl…
ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning
Eric Nazarenus, Chuqiao Li, Yannan He +3
We present ActionPlan, a unified motion diffusion framework that bridges real-time streaming with high-quality offline generation within a single model. The core idea is to introdu…
Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation
Jonas Ernst, Wolfgang Boettcher, Lukas Hoyer +2
We present Rewis3d, a framework that leverages recent advances in feed-forward 3D reconstruction to significantly improve weakly supervised semantic segmentation on 2D images. Obta…
SceneTok: A Compressed, Diffusable Token Space for 3D Scenes
Mohammad Asim, Christopher Wewer, Jan Eric Lenssen
We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens. Existing approaches for 3D scene representation…
SemanticNVS: Improving Semantic Scene Understanding in Generative Novel View Synthesis
Xinya Chen, Christopher Wewer, Jiahao Xie +2
We present SemanticNVS, a camera-conditioned multi-view diffusion model for novel view synthesis (NVS), which improves generation quality and consistency by integrating pre-trained…
MoonSeg3R: Monocular Online Zero-Shot Segment Anything in 3D with Reconstructive Foundation Priors
Zhipeng Du, Duolikun Danier, Jan Eric Lenssen +1
In this paper, we focus on online zero-shot monocular 3D instance segmentation, a novel practical setting where existing approaches fail to perform because they rely on posed RGB-D…
MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
Yannan He, Garvita Tiwari, Xiaohan Zhang +4
We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diff…
AnyUp: Universal Feature Upsampling
Thomas Wimmer, Prune Truong, Marie-Julie Rakotosaona +4
We introduce AnyUp, a method for feature upsampling that can be applied to any vision feature at any resolution, without encoder-specific training. Existing learning-based upsample…
RefAM: Attention Magnets for Zero-Shot Referral Segmentation
Anna Kukleva, Enis Simsar, Alessio Tonioni +4
Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models, often at the cost of additional…
Solving Inverse Problems with FLAIR
Julius Erbach, Dominik Narnhofer, Andreas Dombos +3
Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their imp…
Spatial Reasoning with Denoising Models
Christopher Wewer, Bart Pogodzinski, Bernt Schiele +1
We introduce Spatial Reasoning Models (SRMs), a framework to perform reasoning over sets of continuous variables via denoising generative models. SRMs infer continuous representati…
PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation
Xinting Hu, Haoran Wang, Jan Eric Lenssen +1
We introduce PersonaHOI, a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion (PFD) model to generate identity-consis…
MEt3R: Measuring Multi-View Consistency in Generated Images
Mohammad Asim, Christopher Wewer, Thomas Wimmer +2
We introduce MEt3R, a metric for multi-view consistency in generated images. Large-scale generative models for multi-view image generation are rapidly advancing the field of 3D inf…
SLayR: Scene Layout Generation with Rectified Flow
Cameron Braunstein, Hevra Petekkaya, Jan Eric Lenssen +2
We introduce SLayR, Scene Layout Generation with Rectified flow, a novel transformer-based model for text-to-layout generation which can then be paired with existing layout-to-imag…
Spurfies: Sparse Surface Reconstruction using Local Geometry Priors
Kevin Raj, Christopher Wewer, Raza Yunus +2
We introduce Spurfies, a novel method for sparse-view surface reconstruction that disentangles appearance and geometry information to utilize local geometry priors trained on synth…
InterTrack: Tracking Human Object Interaction without Object Templates
Xianghui Xie, Jan Eric Lenssen, Gerard Pons-Moll
Tracking human object interaction from videos is important to understand human behavior from the rapidly growing stream of video data. Previous video-based methods require predefin…
Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets
Wolfgang Boettcher, Lukas Hoyer, Ozan Unal +2
In this work, we introduce Scribbles for All, a label and training data generation algorithm for semantic segmentation trained on scribble labels. Training or fine-tuning semantic…
Improving 2D Feature Representations by 3D-Aware Fine-Tuning
Yuanwen Yue, Anurag Das, Francis Engelmann +2
Current visual foundation models are trained purely on unstructured 2D data, limiting their understanding of 3D structure of objects and scenes. In this work, we show that fine-tun…
Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors
Soumava Paul, Christopher Wewer, Bernt Schiele +1
We aim to tackle sparse-view reconstruction of a 360 3D scene using priors from latent diffusion models (LDM). The sparse-view setting is ill-posed and underconstrained, especially…
GEARS: Local Geometry-aware Hand-object Interaction Synthesis
Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen +1
Generating realistic hand motion sequences in interaction with objects has gained increasing attention with the growing interest in digital humans. Prior work has illustrated the e…
NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors
Yannan He, Garvita Tiwari, Tolga Birdal +2
Faithfully modeling the space of articulations is a crucial task that allows recovery and generation of realistic poses, and remains a notorious challenge. To this end, we introduc…