Publications (86)
Towards Open-Set Object Detection and Discovery
Jiyang Zheng, Weihao Li, Jie Hong +2
With the human pursuit of knowledge, open-set object detection (OSOD) has been designed to identify unknown objects in a dynamic world. However, an issue with the current setting i…
Question-Agnostic Attention for Visual Question Answering
Moshiur R Farazi, Salman H Khan, Nick Barnes
Visual Question Answering (VQA) models employ attention mechanisms to discover image locations that are most relevant for answering a specific question. For this purpose, several m…
Totally Corrective Multiclass Boosting with Binary Weak Learners
Zhihui Hao, Chunhua Shen, Nick Barnes +1
In this work, we propose a new optimization framework for multiclass boosting learning. In the literature, AdaBoost.MO and AdaBoost.ECC are the two successful multiclass boosting a…
Learning structure-aware semantic segmentation with image-level supervision
Jiawei Liu, Jing Zhang, Yicong Hong +1
Compared with expensive pixel-wise annotations, image-level labels make it possible to learn semantic segmentation in a weakly-supervised manner. Within this pipeline, the class ac…
Dense-Resolution Network for Point Cloud Classification and Segmentation
Shi Qiu, Saeed Anwar, Nick Barnes
Point cloud analysis is attracting attention from Artificial Intelligence research since it can be widely used in applications such as robotics, Augmented Reality, self-driving. Ho…
Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoning
Ge-Peng Ji, Jingyi Liu, Deng-Ping Fan +2
In this study, we present Colon-X, an open initiative aimed at advancing multimodal intelligence in colonoscopy. We begin by constructing ColonVQA, the most comprehensive multimoda…
Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion
Shi Qiu, Saeed Anwar, Nick Barnes
Given the prominence of current 3D sensors, a fine-grained analysis on the basic point cloud data is worthy of further investigation. Particularly, real point cloud scenes can intu…
Perceptually Consistent Color-to-Gray Image Conversion
Shaodi You, Nick Barnes, Janine Walker
In this paper, we propose a color to grayscale image conversion algorithm (C2G) that aims to preserve the perceptual properties of the color image as much as possible. To this end,…
Generalised Co-Salient Object Detection
Jiawei Liu, Jing Zhang, Ruikai Cui +3
We propose a new setting that relaxes an assumption in the conventional Co-Salient Object Detection (CoSOD) setting by allowing the presence of "noisy images" which do not show the…
Robust normalizing flows using Bernstein-type polynomials
Sameera Ramasinghe, Kasun Fernando, Salman Khan +1
Modeling real-world distributions can often be challenging due to sample data that are subjected to perturbations, e.g., instrumentation errors, or added random noise. Since flow m…
NoiseSDF2NoiseSDF: Learning Clean Neural Fields from Noisy Supervision
Tengkai Wang, Weihao Li, Ruikai Cui +2
Reconstructing accurate implicit surface representations from point clouds remains a challenging task, particularly when data is captured using low-quality scanning devices. These…
Efficient Gaussian Process Model on Class-Imbalanced Datasets for Generalized Zero-Shot Learning
Changkun Ye, Nick Barnes, Lars Petersson +1
Zero-Shot Learning (ZSL) models aim to classify object classes that are not seen during the training process. However, the problem of class imbalance is rarely discussed, despite i…
Rethinking Polyp Segmentation from an Out-of-Distribution Perspective
Ge-Peng Ji, Jing Zhang, Dylan Campbell +2
Unlike existing fully-supervised approaches, we rethink colorectal polyp segmentation from an out-of-distribution perspective with a simple but effective self-supervised learning a…
RGB-D Saliency Detection via Cascaded Mutual Information Minimization
Jing Zhang, Deng-Ping Fan, Yuchao Dai +4
Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. In this paper, we introduce a novel multi-stage cascad…
Asymmetric Totally-corrective Boosting for Real-time Object Detection
Peng Wang, Chunhua Shen, Nick Barnes +2
Real-time object detection is one of the core problems in computer vision. The cascade boosting framework proposed by Viola and Jones has become the standard for this problem. In t…
Confidence-Aware Learning for Camouflaged Object Detection
Jiawei Liu, Jing Zhang, Nick Barnes
Confidence-aware learning is proven as an effective solution to prevent networks becoming overconfident. We present a confidence-aware camouflaged object detection framework using…
PnP-3D: A Plug-and-Play for 3D Point Clouds
Shi Qiu, Saeed Anwar, Nick Barnes
With the help of the deep learning paradigm, many point cloud networks have been invented for visual analysis. However, there is great potential for development of these networks s…
Spectral-GANs for High-Resolution 3D Point-cloud Generation
Sameera Ramasinghe, Salman Khan, Nick Barnes +1
Point-clouds are a popular choice for vision and graphics tasks due to their accurate shape description and direct acquisition from range-scanners. This demands the ability to synt…
FreeVPS: Repurposing Training-Free SAM2 for Generalizable Video Polyp Segmentation
Qiang Hu, Ying Zhou, Gepeng Ji +3
Existing video polyp segmentation (VPS) paradigms usually struggle to balance between spatiotemporal modeling and domain generalization, limiting their applicability in real clinic…
Transferable Attack for Semantic Segmentation
Mengqi He, Jing Zhang, Zhaoyuan Yang +3
We analysis performance of semantic segmentation models wrt. adversarial attacks, and observe that the adversarial examples generated from a source model fail to attack the target…
Measuring and Modeling Uncertainty Degree for Monocular Depth Estimation
Mochu Xiang, Jing Zhang, Nick Barnes +1
Effectively measuring and modeling the reliability of a trained model is essential to the real-world deployment of monocular depth estimation (MDE) models. However, the intrinsic i…
Learning To Segment Dominant Object Motion From Watching Videos
Sahir Shrestha, Mohammad Ali Armin, Hongdong Li +1
Existing deep learning based unsupervised video object segmentation methods still rely on ground-truth segmentation masks to train. Unsupervised in this context only means that no…
Dense Uncertainty Estimation
Jing Zhang, Yuchao Dai, Mochu Xiang +7
Deep neural networks can be roughly divided into deterministic neural networks and stochastic neural networks.The former is usually trained to achieve a mapping from input space to…
Uncertainty-Aware Deep Calibrated Salient Object Detection
Jing Zhang, Yuchao Dai, Xin Yu +3
Existing deep neural network based salient object detection (SOD) methods mainly focus on pursuing high network accuracy. However, those methods overlook the gap between network ac…
A Comprehensive Overview of Large Language Models
Humza Naveed, Asad Ullah Khan, Shi Qiu +6
Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of r…
Volumetric Convolution: Automatic Representation Learning in Unit Ball
Sameera Ramasinghe, Salman Khan, Nick Barnes
Convolution is an efficient technique to obtain abstract feature representations using hierarchical layers in deep networks. Although performing convolution in Euclidean geometries…
Generative Transformer for Accurate and Reliable Salient Object Detection
Yuxin Mao, Jing Zhang, Zhexiong Wan +6
Transformer, which originates from machine translation, is particularly powerful at modeling long-range dependencies. Currently, the transformer is making revolutionary progress in…
A Deep Journey into Super-resolution: A survey
Saeed Anwar, Salman Khan, Nick Barnes
Deep convolutional networks based super-resolution is a fast-growing field with numerous practical applications. In this exposition, we extensively compare 30+ state-of-the-art sup…
Open Set Label Shift with Test Time Out-of-Distribution Reference
Changkun Ye, Russell Tsuchida, Lars Petersson +1
Open set label shift (OSLS) occurs when label distributions change from a source to a target distribution, and the target distribution has an additional out-of-distribution (OOD) c…
Learning RGB-D Salient Object Detection using background enclosure, depth contrast, and top-down features
Riku Shigematsu, David Feng, Shaodi You +1
Recently, deep Convolutional Neural Networks (CNN) have demonstrated strong performance on RGB salient object detection. Although, depth information can help improve detection resu…
Weakly-supervised Contrastive Learning for Unsupervised Object Discovery
Yunqiu Lv, Jing Zhang, Nick Barnes +1
Unsupervised object discovery (UOD) refers to the task of discriminating the whole region of objects from the background within a scene without relying on labeled datasets, which b…
Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction
Jing Zhang, Jianwen Xie, Nick Barnes +1
Vision transformer networks have shown superiority in many computer vision tasks. In this paper, we take a step further by proposing a novel generative vision transformer with late…
Conditional Generative Modeling via Learning the Latent Space
Sameera Ramasinghe, Kanchana Ranasinghe, Salman Khan +2
Although deep learning has achieved appealing results on several machine learning tasks, most of the models are deterministic at inference, limiting their application to single-mod…
Adversarial Training of Variational Auto-encoders for High Fidelity Image Generation
Salman H. Khan, Munawar Hayat, Nick Barnes
Variational auto-encoders (VAEs) provide an attractive solution to image generation problem. However, they tend to produce blurred and over-smoothed images due to their dependence…
Deep Texture and Structure Aware Filtering Network for Image Smoothing
Kaiyue Lu, Shaodi You, Nick Barnes
Image smoothing is a fundamental task in computer vision, that aims to retain salient structures and remove insignificant textures. In this paper, we aim to address the fundamental…
Rethinking conditional GAN training: An approach using geometrically structured latent manifolds
Sameera Ramasinghe, Moshiur Farazi, Salman Khan +2
Conditional GANs (cGAN), in their rudimentary form, suffer from critical drawbacks such as the lack of diversity in generated outputs and distortion between the latent and output m…
AusSmoke meets MultiNatSmoke: a fully-labelled diverse smoke segmentation dataset
Weihao Li, Hongjin Zhao, Gao Zhu +4
Wildfires are an escalating global concern due to the devastating impacts on the environment, economy, and human health, with notable incidents such as the 2019-2020 Australian bus…
PU-Transformer: Point Cloud Upsampling Transformer
Shi Qiu, Saeed Anwar, Nick Barnes
Given the rapid development of 3D scanners, point clouds are becoming popular in AI-driven machines. However, point cloud data is inherently sparse and irregular, causing significa…
Semi-supervised Salient Object Detection with Effective Confidence Estimation
Jiawei Liu, Jing Zhang, Nick Barnes
The success of existing salient object detection models relies on a large pixel-wise labeled training dataset, which is time-consuming and expensive to obtain. We study semi-superv…
Simultaneously Localize, Segment and Rank the Camouflaged Objects
Yunqiu Lv, Jing Zhang, Yuchao Dai +4
Camouflage is a key defence mechanism across species that is critical to survival. Common strategies for camouflage include background matching, imitating the color and pattern of…
NumGrad-Pull: Numerical Gradient Guided Tri-plane Representation for Surface Reconstruction from Point Clouds
Ruikai Cui, Binzhu Xie, Shi Qiu +3
Reconstructing continuous surfaces from unoriented and unordered 3D points is a fundamental challenge in computer vision and graphics. Recent advancements address this problem by t…
Frontiers in Intelligent Colonoscopy
Ge-Peng Ji, Jingyi Liu, Peng Xu +4
Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer. This study investigates the frontiers of intelligent colonoscopy techniques and their pr…
All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation
Weixuan Sun, Yanhao Zhang, Zhen Qin +5
In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation…
CED: Color Event Camera Dataset
Cedric Scheerlinck, Henri Rebecq, Timo Stoffregen +3
Event cameras are novel, bio-inspired visual sensors, whose pixels output asynchronous and independent timestamped spikes at local intensity changes, called 'events'. Event cameras…
GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation
Weixuan Sun, Jing Zhang, Zheyuan Liu +2
Weakly Supervised Semantic Segmentation (WSSS) is challenging, particularly when image-level labels are used to supervise pixel level prediction. To bridge their gap, a Class Activ…
Attention Based Real Image Restoration
Saeed Anwar, Nick Barnes, Lars Petersson
Deep convolutional neural networks perform better on images containing spatially invariant degradations, also known as synthetic degradations; however, their performance is limited…
SDI-Paste: Synthetic Dynamic Instance Copy-Paste for Video Instance Segmentation
Sahir Shrestha, Weihao Li, Gao Zhu +1
Data augmentation methods such as Copy-Paste have been studied as effective ways to expand training datasets while incurring minimal costs. While such methods have been extensively…
Geometric Back-projection Network for Point Cloud Classification
Shi Qiu, Saeed Anwar, Nick Barnes
As the basic task of point cloud analysis, classification is fundamental but always challenging. To address some unsolved problems of existing methods, we propose a network that ca…
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
Jing Zhang, Deng-Ping Fan, Yuchao Dai +4
In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detecti…
3D Guided Weakly Supervised Semantic Segmentation
Weixuan Sun, Jing Zhang, Nick Barnes
Pixel-wise clean annotation is necessary for fully-supervised semantic segmentation, which is laborious and expensive to obtain. In this paper, we propose a weakly supervised 2D se…
Blended Convolution and Synthesis for Efficient Discrimination of 3D Shapes
Sameera Ramasinghe, Salman Khan, Nick Barnes +1
Existing networks directly learn feature representations on 3D point clouds for shape analysis. We argue that 3D point clouds are highly redundant and hold irregular (permutation-i…
SmokeBench: Evaluating Multimodal Large Language Models for Wildfire Smoke Detection
Tianye Qi, Weihao Li, Nick Barnes
Wildfire smoke is transparent, amorphous, and often visually confounded with clouds, making early-stage detection particularly challenging. In this work, we introduce a benchmark,…
Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection
Jing Zhang, Jianwen Xie, Nick Barnes
In this paper, we propose a noise-aware encoder-decoder framework to disentangle a clean saliency predictor from noisy training examples, where the noisy labels are generated by un…
Any-Shot Object Detection
Shafin Rahman, Salman Khan, Nick Barnes +1
Previous work on novel object detection considers zero or few-shot settings where none or few examples of each category are available for training. In real world scenarios, it is l…
LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image
Ruikai Cui, Xibin Song, Weixuan Sun +8
Large Reconstruction Models have made significant strides in the realm of automated 3D content generation from single or multiple input images. Despite their success, these models…
Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation
Weixuan Sun, Jing Zhang, Nick Barnes
Image-level weakly supervised semantic segmentation (WSSS) relies on class activation maps (CAMs) for pseudo labels generation. As CAMs only highlight the most discriminative regio…
Representation Learning on Unit Ball with 3D Roto-Translational Equivariance
Sameera Ramasinghe, Salman Khan, Nick Barnes +1
Convolution is an integral operation that defines how the shape of one function is modified by another function. This powerful concept forms the basis of hierarchical feature learn…
Densely Residual Laplacian Super-Resolution
Saeed Anwar, Nick Barnes
Super-Resolution convolutional neural networks have recently demonstrated high-quality restoration for single images. However, existing algorithms often require very deep architect…
Learning Gaussian Representation for Eye Fixation Prediction
Peipei Song, Jing Zhang, Piotr Koniusz +1
Existing eye fixation prediction methods perform the mapping from input images to the corresponding dense fixation maps generated from raw fixation points. However, due to the stoc…
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
Weakly Supervised Video Salient Object Detection
Wangbo Zhao, Jing Zhang, Long Li +3
Significant performance improvement has been achieved for fully-supervised video salient object detection with the pixel-wise labeled training datasets, which are time-consuming an…
Attention Guided Semantic Relationship Parsing for Visual Question Answering
Moshiur Farazi, Salman Khan, Nick Barnes
Humans explain inter-object relationships with semantic labels that demonstrate a high-level understanding required to perform complex Vision-Language tasks such as Visual Question…
PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning
Jie Hong, Shi Qiu, Weihao Li +4
Point cloud learning is receiving increasing attention. However, most existing point cloud models lack the practical ability to deal with the unavoidable presence of unknown object…
Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Weixuan Sun, Jiayi Zhang, Jianyuan Wang +6
Self-supervised audio-visual source localization aims to locate sound-source objects in video frames without extra annotations. Recent methods often approach this goal with the hel…
Accuracy vs. Complexity: A Trade-off in Visual Question Answering Models
Moshiur R. Farazi, Salman H. Khan, Nick Barnes
Visual Question Answering (VQA) has emerged as a Visual Turing Test to validate the reasoning ability of AI agents. The pivot to existing VQA models is the joint embedding that is…
Unsupervised Primitive Discovery for Improved 3D Generative Modeling
Salman H. Khan, Yulan Guo, Munawar Hayat +1
3D shape generation is a challenging problem due to the high-dimensional output space and complex part configurations of real-world objects. As a result, existing algorithms experi…
Totally Corrective Boosting for Regularized Risk Minimization
Chunhua Shen, Hanxi Li, Nick Barnes
Consideration of the primal and dual problems together leads to important new insights into the characteristics of boosting algorithms. In this work, we propose a general framework…
Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model
Jing Zhang, Yuchao Dai, Mehrtash Harandi +3
Uncertainty estimation has been extensively studied in recent literature, which can usually be classified as aleatoric uncertainty and epistemic uncertainty. In current aleatoric u…
Uncertainty Inspired RGB-D Saliency Detection
Jing Zhang, Deng-Ping Fan, Yuchao Dai +4
We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models tr…
From Known to the Unknown: Transferring Knowledge to Answer Questions about Novel Visual and Semantic Concepts
Moshiur R Farazi, Salman H Khan, Nick Barnes
Current Visual Question Answering (VQA) systems can answer intelligent questions about `Known' visual content. However, their performance drops significantly when questions about v…
A Systematic Evaluation: Fine-Grained CNN vs. Traditional CNN Classifiers
Saeed Anwar, Nick Barnes, Lars Petersson
To make the best use of the underlying minute and subtle differences, fine-grained classifiers collect information about inter-class variations. The task is very challenging due to…
Energy-Based Residual Latent Transport for Unsupervised Point Cloud Completion
Ruikai Cui, Shi Qiu, Saeed Anwar +2
Unsupervised point cloud completion aims to infer the whole geometry of a partial object observation without requiring partial-complete correspondence. Differing from existing dete…
Polarity Loss for Zero-shot Object Detection
Shafin Rahman, Salman Khan, Nick Barnes
Conventional object detection models require large amounts of training data. In comparison, humans can recognize previously unseen objects by merely knowing their semantic descript…
P2C: Self-Supervised Point Cloud Completion from Single Partial Clouds
Ruikai Cui, Shi Qiu, Saeed Anwar +4
Point cloud completion aims to recover the complete shape based on a partial observation. Existing methods require either complete point clouds or multiple partial observations of…
Asynchronous Spatial Image Convolutions for Event Cameras
Cedric Scheerlinck, Nick Barnes, Robert Mahony
Spatial convolution is arguably the most fundamental of 2D image processing operations. Conventional spatial image convolution can only be applied to a conventional image, that is,…
Continuous-time Intensity Estimation Using Event Cameras
Cedric Scheerlinck, Nick Barnes, Robert Mahony
Event cameras provide asynchronous, data-driven measurements of local temporal contrast over a large dynamic range with extremely high temporal resolution. Conventional cameras cap…
Recursive Training for Zero-Shot Semantic Segmentation
Ce Wang, Moshiur Farazi, Nick Barnes
General purpose semantic segmentation relies on a backbone CNN network to extract discriminative features that help classify each image pixel into a 'seen' object class (ie., the o…
Reducing the Sim-to-Real Gap for Event Cameras
Timo Stoffregen, Cedric Scheerlinck, Davide Scaramuzza +4
Event cameras are paradigm-shifting novel sensors that report asynchronous, per-pixel brightness changes called 'events' with unparalleled low latency. This makes them ideal for hi…
Towards Deeper Understanding of Camouflaged Object Detection
Yunqiu Lv, Jing Zhang, Yuchao Dai +3
Preys in the wild evolve to be camouflaged to avoid being recognized by predators. In this way, camouflage acts as a key defence mechanism across species that is critical to surviv…
Energy-Based Generative Cooperative Saliency Prediction
Jing Zhang, Jianwen Xie, Zilong Zheng +1
Conventional saliency prediction models typically learn a deterministic mapping from an image to its saliency map, and thus fail to explain the subjective nature of human attention…
Vicinity Vision Transformer
Weixuan Sun, Zhen Qin, Hui Deng +7
Vision transformers have shown great success on numerous computer vision tasks. However, its central component, softmax attention, prohibits vision transformers from scaling up to…
An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems
Weixuan Sun, Zheyuan Liu, Yanhao Zhang +2
The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the appl…
Transmission-Guided Bayesian Generative Model for Smoke Segmentation
Siyuan Yan, Jing Zhang, Nick Barnes
Smoke segmentation is essential to precisely localize wildfire so that it can be extinguished in an early phase. Although deep neural networks have achieved promising results on im…
Model Calibration in Dense Classification with Adaptive Label Perturbation
Jiawei Liu, Changkun Ye, Shan Wang +4
For safety-related applications, it is crucial to produce trustworthy deep neural networks whose prediction is associated with confidence that can represent the likelihood of corre…
Real Image Denoising with Feature Attention
Saeed Anwar, Nick Barnes
Deep convolutional neural networks perform better on images containing spatially invariant noise (synthetic noise); however, their performance is limited on real-noisy photographs…
An Energy-Based Prior for Generative Saliency
Jing Zhang, Jianwen Xie, Nick Barnes +1
We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. The energy-based prior model is defined on the la…