2 citations · 5 across the 22 of their papers we have counts for
23 papers
TILT: Model-Intrinsic Reward Alignment For Compositional Diffusion
Debottam Dutta, Jaehoon Hahm, Jianchong Chen +1
Consider conditional generation where is a prompt composed of multiple concepts . Diffusion models often struggle with compositional…
Discrete Langevin-Inspired Posterior Sampling
Chaitanya Amballa, Sattwik Basu, Jorge Vančo Sampedro +1
We study posterior sampling for inverse problems in discrete state spaces using discrete diffusion models as generative priors. While continuous diffusion models have become widely…
Dependency-Aware Discrete Diffusion for Scene Graph Generation
Rajalaxmi Rajagopalan, Romit Roy Choudhury
Scene graphs (SGs) represent objects and their relationships as structured graphs, enabling applications in image generation, robotics, and 3D understanding. Recent work suggests t…
Unified Diffusion Refinement for Multi-Channel Speech Enhancement and Separation
Zhongweiyang Xu, Ashutosh Pandey, Juan Azcarreta +4
We propose Uni-ArrayDPS, a novel diffusion-based refinement framework for unified multi-channel speech enhancement and separation. Existing methods for multi-channel speech enhance…
Personalized Image Generation via Human-in-the-loop Bayesian Optimization
Rajalaxmi Rajagopalan, Debottam Dutta, Yu-Lin Wei +1
Imagine Alice has a specific image in her mind, say, the view of the street in which she grew up during her childhood. To generate that exact image, she guides a generativ…
AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Xilin Jiang, Qiaolin Wang, Junkai Wu +30
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand suc…
Zero-shot Human Pose Estimation using Diffusion-based Inverse solvers
Sahil Bhandary Karnoor, Romit Roy Choudhury
Pose estimation refers to tracking a human's full body posture, including their head, torso, arms, and legs. The problem is challenging in practical settings where the number of bo…
Learning Energy-based Variational Latent Prior for VAEs
Debottam Dutta, Chaitanya Amballa, Zhongweiyang Xu +2
Variational Auto-Encoders (VAEs) are known to generate blurry and inconsistent samples. One reason for this is the "prior hole" problem. A prior hole refers to regions that have hi…
Contrastive Diffusion Guidance for Spatial Inverse Problems
Sattwik Basu, Chaitanya Amballa, Zhongweiyang Xu +3
We consider a class of inverse problems characterized by forward operators that are partially specified, non-smooth, and non-differentiable. Although generative inverse solvers hav…
Steer Away From Mode Collisions: Improving Composition In Diffusion Models
Debottam Dutta, Jianchong Chen, Rajalaxmi Rajagopalan +2
We propose to improve multi-concept prompt fidelity in text-to-image diffusion models. We begin with common failure cases - prompts like "a cat and a dog" that sometimes yields ima…
Explicit Context-Driven Neural Acoustic Modeling for High-Fidelity RIR Generation
Chen Si, Qianyi Wu, Chaitanya Amballa +1
Realistic sound simulation plays a critical role in many applications. A key element in sound simulation is the room impulse response (RIR), which characterizes how sound propagate…
Unsupervised Multi-channel Speech Dereverberation via Diffusion
Yulun Wu, Zhongweiyang Xu, Jianchong Chen +2
We consider the problem of multi-channel single-speaker blind dereverberation, where multi-channel mixtures are used to recover the clean anechoic speech. To solve this problem, we…
Kernel Learning for Sample Constrained Black-Box Optimization
Rajalaxmi Rajagopalan, Yu-Lin Wei, Romit Roy Choudhury
Black box optimization (BBO) focuses on optimizing unknown functions in high-dimensional spaces. In many applications, sampling the unknown function is expensive, imposing a tight…
Sample-Constrained Black Box Optimization for Audio Personalization
Rajalaxmi Rajagopalan, Yu-Lin Wei, Romit Roy Choudhury
We consider the problem of personalizing audio to maximize user experience. Briefly, we aim to find a filter , which applied to any music or speech, will maximize the user's s…
ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Zhongweiyang Xu, Xulin Fan, Zhong-Qiu Wang +2
Blind Speech Separation (BSS) aims to separate multiple speech sources from audio mixtures recorded by a microphone array. The problem is challenging because it is a blind inverse…
Can NeRFs See without Cameras?
Chaitanya Amballa, Sattwik Basu, Yu-Lin Wei +3
Neural Radiance Fields (NeRFs) have been remarkably successful at synthesizing novel views of 3D scenes by optimizing a volumetric scene function. This scene function models how op…
Estimating Multi-chirp Parameters using Curvature-guided Langevin Monte Carlo
Sattwik Basu, Debottam Dutta, Yu-Lin Wei +1
This paper considers the problem of estimating chirp parameters from a noisy mixture of chirps. While a rich body of work exists in this area, challenges remain when extending thes…
Multi-Source Music Generation with Latent Diffusion
Zhongweiyang Xu, Debottam Dutta, Yu-Lin Wei +1
Most music generation models directly generate a single music mixture. To allow for more flexible and controllable generation, the Multi-Source Diffusion Model (MSDM) has been prop…
Reducing Opinion Echo-Chambers by Intelligent Placement of Moderate-Minded Agents
Prithwish Jana, Romit Roy Choudhury, Niloy Ganguly
In the era of social media, people frequently share their own opinions online on various issues and also in the way, get exposed to others' opinions. Be it for selective exposure o…
Learning to Separate Voices by Spatial Regions
Zhongweiyang Xu, Romit Roy Choudhury
We consider the problem of audio voice separation for binaural applications, such as earphones and hearing aids. While today's neural networks perform remarkably well (separating $…
RoSS: Utilizing Robotic Rotation for Audio Source Separation
Hyungjoo Seo, Sahil Bhandary Karnoor, Romit Roy Choudhury
This paper considers the problem of audio source separation where the goal is to isolate a target audio signal (say Alice's speech) from a mixture of multiple interfering signals (…
Inferring Facing Direction from Voice Signals
Yu-Lin Wei, Rui Li, Abhinav Mehrotra +2
Consider a home or office where multiple devices are running voice assistants (e.g., TVs, lights, ovens, refrigerators, etc.). A human user turns to a particular device and gives a…
Estimating Angle of Arrival (AoA) of multiple Echoes in a Steering Vector Space
Yu-Lin Wei, Romit Roy Choudhury
Consider a microphone array, such as those present in Amazon Echos, conference phones, or self-driving cars. One of the goals of these arrays is to decode the angles in which acous…