207 citations · 529 across the 10 of their papers we have counts for
12 papers
Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
Cusuh Ham, James Hays, Jingwan Lu +3
We present multimodal conditioning modules (MCM) for enabling conditional image synthesis using pretrained diffusion models. Previous multimodal synthesis works rely on training ne…
SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model
Shaoan Xie, Zhifei Zhang, Zhe Lin +2
Generic image inpainting aims to complete a corrupted image by borrowing surrounding information, which barely generates novel content. By contrast, multi-modal inpainting provides…
CharacterGAN: Few-Shot Keypoint Character Animation and Reposing
Tobias Hinz, Matthew Fisher, Oliver Wang +2
We introduce CharacterGAN, a generative model that can be trained on only a few samples (8 - 15) of a given character. Our model generates novel poses based on keypoint locations,…
Adversarial Text-to-Image Synthesis: A Review
Stanislav Frolov, Tobias Hinz, Federico Raue +2
With the advent of generative adversarial networks, synthesizing images from textual descriptions has recently become an active research area. It is a flexible and intuitive way fo…
Crossmodal Language Grounding in an Embodied Neurocognitive Model
Stefan Heinrich, Yuan Yao, Tobias Hinz +5
Human infants are able to acquire natural language seemingly easily at an early age. Their language learning seems to occur simultaneously with learning other cognitive functions a…
Improved Techniques for Training Single-Image GANs
Tobias Hinz, Matthew Fisher, Oliver Wang +1
Recently there has been an interest in the potential of learning generative models from a single image, as opposed to from a large dataset. This task is of practical significance,…
Semantic Object Accuracy for Generative Text-to-Image Synthesis
Tobias Hinz, Stefan Heinrich, Stefan Wermter
Generative adversarial networks conditioned on textual image descriptions are capable of generating realistic-looking images. However, current methods still struggle to generate im…
Evaluating Defensive Distillation For Defending Text Processing Neural Networks Against Adversarial Examples
Marcus Soll, Tobias Hinz, Sven Magg +1
Adversarial examples are artificially modified input samples which lead to misclassifications, while not being detectable by humans. These adversarial examples are a challenge for…
Generating Multiple Objects at Spatially Distinct Locations
Tobias Hinz, Stefan Heinrich, Stefan Wermter
Recent improvements to Generative Adversarial Networks (GANs) have made it possible to generate realistic images in high resolution based on natural language descriptions such as i…
Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks
Tobias Hinz, Nicolás Navarro-Guerrero, Sven Magg +1
Most learning algorithms require the practitioner to manually set the values of many hyperparameters before the learning process can begin. However, with modern algorithms, the eva…
Image Generation and Translation with Disentangled Representations
Tobias Hinz, Stefan Wermter
Generative models have made significant progress in the tasks of modeling complex data distributions such as natural images. The introduction of Generative Adversarial Networks (GA…
Inferencing Based on Unsupervised Learning of Disentangled Representations
Tobias Hinz, Stefan Wermter
Combining Generative Adversarial Networks (GANs) with encoders that learn to encode data points has shown promising results in learning data representations in an unsupervised way.…