224 citations · 429 across the 42 of their papers we have counts for
43 papers
CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models
He Liang, Chenyang Ma, Yiming Zhang +4
Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world…
Mitigating Cognitive Bias in RLHF by Altering Rationality
Tiffany Horter, Andrew Markham, Niki Trigoni +1
How can we make models robust to even imperfect human feedback? In reinforcement learning from human feedback (RLHF), human preferences over model outputs are used to train a rewar…
CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding
Chenyang Ma, Kai Lu, Guangyu Yang +6
Current work on robot failure detection and correction typically operates in a post hoc manner, analyzing errors and applying corrections only after failures occur. This work intro…
COOPERA: Continual Open-Ended Human-Robot Assistance
Chenyang Ma, Kai Lu, Ruta Desai +3
To understand and collaborate with humans, robots must account for individual human traits, habits, and activities over time. However, most robotic assistants lack these abilities,…
Data Factory with Minimal Human Effort Using VLMs
Jiaojiao Ye, Jiaxing Zhong, Qian Xie +3
Generating enough and diverse data through augmentation offers an efficient solution to the time-consuming and labour-intensive process of collecting and annotating pixel-wise imag…
Efficient and Microphone-Fault-Tolerant 3D Sound Source Localization
Yiyuan Yang, Shitong Xu, Niki Trigoni +1
Sound source localization (SSL) is a critical technology for determining the position of sound sources in complex environments. However, existing methods face challenges such as hi…
Target Speaker Extraction through Comparing Noisy Positive and Negative Audio Enrollments
Shitong Xu, Yiyuan Yang, Niki Trigoni +1
Target speaker extraction focuses on isolating a specific speaker's voice from an audio mixture containing multiple speakers. To provide information about the target speaker's iden…
SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera
Yuhang He, Sangyun Shin, Anoop Cherian +2
Accurately localizing 3D sound sources and estimating their semantic labels -- where the sources may not be visible, but are assumed to lie on the physical surface of objects in th…
Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis
Qianyi Deng, Oishi Deb, Amir Patel +4
Animal pose estimation (APE) aims to locate the animal body parts using a diverse array of sensor and modality inputs (e.g. RGB cameras, LiDAR, infrared, IMU, acoustic and language…
MambaLoc: Efficient Camera Localisation via State Space Model
Jialu Wang, Kaichen Zhou, Andrew Markham +1
Location information is pivotal for the automation and intelligence of terminal devices and edge-cloud IoT systems, such as autonomous vehicles and augmented reality. However, achi…
SPEAR: Receiver-to-Receiver Acoustic Neural Warping Field
Yuhang He, Shitong Xu, Jia-Xing Zhong +3
We present SPEAR, a continuous receiver-to-receiver acoustic neural warping field for spatial acoustic effects prediction in an acoustic 3D space with a single stationary audio sou…
Pre-training Feature Guided Diffusion Model for Speech Enhancement
Yiyuan Yang, Niki Trigoni, Andrew Markham
Speech enhancement significantly improves the clarity and intelligibility of speech in noisy environments, improving communication and listening experiences. In this paper, we intr…
Dusk Till Dawn: Self-supervised Nighttime Stereo Depth Estimation using Visual Foundation Models
Madhu Vankadari, Samuel Hodgson, Sangyun Shin +2
Self-supervised depth estimation algorithms rely heavily on frame-warping relationships, exhibiting substantial performance degradation when applied in challenging circumstances, s…
ZeST: Zero-Shot Material Transfer from a Single Image
Ta-Ying Cheng, Prafull Sharma, Andrew Markham +2
We propose ZeST, a method for zero-shot material transfer to an object in the input image given a material exemplar image. ZeST leverages existing diffusion adapters to extract imp…
WSCLoc: Weakly-Supervised Sparse-View Camera Relocalization
Jialu Wang, Kaichen Zhou, Andrew Markham +1
Despite the advancements in deep learning for camera relocalization tasks, obtaining ground truth pose labels required for the training process remains a costly endeavor. While cur…
SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
Chenyang Ma, Kai Lu, Ta-Ying Cheng +2
Current state-of-the-art spatial reasoning-enhanced VLMs are trained to excel at spatial visual question answering (VQA). However, we believe that higher-level 3D-aware tasks, such…
Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition
Chun-Hsiao Yeh, Ta-Ying Cheng, He-Yen Hsieh +6
Recent text-to-image diffusion models are able to learn and synthesize images containing novel, personalized concepts (e.g., their own pets or specific items) with just a few examp…
Learning Continuous 3D Words for Text-to-Image Generation
Ta-Ying Cheng, Matheus Gadelha, Thibault Groueix +4
Current controls over diffusion models (e.g., through text or ControlNet) for image generation fall short in recognizing abstract, continuous attributes like illumination direction…
SoundCount: Sound Counting from Raw Audio with Dyadic Decomposition Neural Network
Yuhang He, Zhuangzhuang Dai, Long Chen +2
In this paper, we study an underexplored, yet important and challenging problem: counting the number of distinct sounds in raw audio characterized by a high degree of polyphonicity…
Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenes
Kaichen Zhou, Jia-Wang Bian, Jian-Qing Zheng +4
Despite advancements in self-supervised monocular depth estimation, challenges persist in dynamic scenarios due to the dependence on assumptions about a static world. In this paper…
Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation
Sangyun Shin, Kaichen Zhou, Madhu Vankadari +2
Coarse-to-fine 3D instance segmentation methods show weak performances compared to recent Grouping-based, Kernel-based and Transformer-based methods. We argue that this is due to t…
3DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasets
Ta-Ying Cheng, Matheus Gadelha, Soren Pirk +4
We present 3DMiner -- a pipeline for mining 3D shapes from challenging large-scale unannotated image datasets. Unlike other unsupervised 3D reconstruction methods, we assume that,…
SSL-Net: A Synergistic Spectral and Learning-based Network for Efficient Bird Sound Classification
Yiyuan Yang, Kaichen Zhou, Niki Trigoni +1
Efficient and accurate bird sound classification is of important for ecology, habitat protection and scientific research, as it plays a central role in monitoring the distribution…
Deep Learning for Visual Localization and Mapping: A Survey
Changhao Chen, Bing Wang, Chris Xiaoxuan Lu +2
Deep learning based localization and mapping approaches have recently emerged as a new research direction and receive significant attentions from both industry and academia. Instea…