204 citations · 213 across the 25 of their papers we have counts for
26 papers
C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs
Jiameng Li, Han Zhou, Matthew B. Blaschko
Multimodal large language models (MLLMs) require huge memory and computational costs, which limits their practical deployment. Post-training quantization (PTQ) techniques offer an…
Bandwidth Selection in Kernel Density Estimation for Model Calibration
Han Zhou, Teodora Popordanoska, Matthew Blaschko
As deep learning models are increasingly deployed in high-stakes applications, providing well-calibrated uncertainty estimates has become as critical as achieving high predictive a…
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
Konstantinos Kontras, Teodora Gagaleska, Thomas Strypsteen +4
A central objective in multimodal learning is to capture synergy: task-relevant information that arises only from the joint use of multiple modalities, and is not available from an…
EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs
Jiameng Li, Minye Wu, Jiezhang Cao +2
Long-form video understanding remains challenging for Video Large Language Models (VideoLLMs), as the dense frame sampling introduces massive visual tokens while sparse sampling ri…
MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs
Jiameng Li, Aleksei Tiulpin, Matthew B. Blaschko
For multimodal large language models (MLLMs), visual information is relatively sparse compared with text. As a result, research on visual pruning emerges for efficient inference. C…
Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion
Mang Ning, Mingxiao Li, Le Zhang +4
In this paper, we study the diffusability (learnability) of variational autoencoders (VAEs) in latent diffusion. We first derive distribution-free bounds on the maximum achievable…
When normalization hallucinates: unseen risks in AI-powered whole slide image processing
Karel Moens, Matthew B. Blaschko, Tinne Tuytelaars +3
Whole slide image (WSI) normalization remains a vital preprocessing step in computational pathology. Increasingly driven by deep learning, these models learn to approximate data di…
CLASH: A Benchmark for Cross-Modal Contradiction Detection
Teodora Popordanoska, Jiameng Li, Matthew B. Blaschko
Contradictory multimodal inputs are common in real-world settings, yet existing benchmarks typically assume input consistency and fail to evaluate cross-modal contradiction detecti…
SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Model
Dongli Xu, Aleksei Tiulpin, Matthew B. Blaschko
Autoregressive (AR) models have emerged as powerful tools for image generation by modeling images as sequences of discrete tokens. While Classifier-Free Guidance (CFG) has been ado…
Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning
Chang Tian, Matthew B. Blaschko, Mingzhe Xing +3
Reinforcement learning (RL) has become a key technique for enhancing the reasoning abilities of large language models (LLMs), with policy-gradient algorithms dominating the post-tr…
Bayesian Optimization over Bounded Domains with the Beta Product Kernel
Huy Hoang Nguyen, Han Zhou, Matthew B. Blaschko +1
Bayesian optimization with Gaussian processes (GP) is commonly used to optimize black-box functions. The Matérn and the Radial Basis Function (RBF) covariance functions are used fr…
Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles
Zifu Wang, Junyi Zhu, Bo Tang +4
The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text-on…
Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Loss
Han Zhou, Sebastian G. Gruber, Teodora Popordanoska +1
Several variants of reweighted risk functionals, such as focal loss, inverse focal loss, and the Area Under the Risk Coverage Curve (AURC), have been proposed for improving model c…
CARE: Confidence-aware Ratio Estimation for Medical Biomarkers
Jiameng Li, Teodora Popordanoska, Aleksei Tiulpin +3
Ratio-based biomarkers (RBBs), such as the proportion of necrotic tissue within a tumor, are widely used in clinical practice to support diagnosis, prognosis, and treatment plannin…
Surgeons vs. Computer Vision: A comparative analysis on surgical phase recognition capabilities
Marco Mezzina, Pieter De Backer, Tom Vercauteren +3
Purpose: Automated Surgical Phase Recognition (SPR) uses Artificial Intelligence (AI) to segment the surgical workflow into its key events, functioning as a building block for effi…
DAVE: Diagnostic benchmark for Audio Visual Evaluation
Gorjan Radevski, Teodora Popordanoska, Matthew B. Blaschko +1
Audio-visual understanding is a rapidly evolving field that seeks to integrate and interpret information from both auditory and visual modalities. Despite recent advances in multi-…
Using Causality for Enhanced Prediction of Web Traffic Time Series
Chang Tian, Mingzhe Xing, Zenglin Shi +3
Predicting web service traffic has significant social value, as it can be applied to various practical scenarios, including but not limited to dynamic resource scaling, load balanc…
A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators
Han Zhou, Jordy Van Landeghem, Teodora Popordanoska +1
The selective classifier (SC) has been proposed for rank based uncertainty thresholding, which could have applications in safety critical areas such as medical diagnostics, autonom…
A Generic Method for Fine-grained Category Discovery in Natural Language Texts
Chang Tian, Matthew B. Blaschko, Wenpeng Yin +3
Fine-grained category discovery using only coarse-grained supervision is a cost-effective yet challenging task. Previous training methods focus on aligning query samples with posit…
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications
Jordy Van Landeghem, Subhajit Maity, Ayan Banerjee +4
This work explores knowledge distillation (KD) for visually-rich document (VRD) applications such as document layout analysis (DLA) and document image classification (DIC). While V…
Multimodal Adaptive Inference for Document Image Classification with Anytime Early Exiting
Omar Hamed, Souhail Bakkali, Marie-Francine Moens +2
This work addresses the need for a balanced approach between performance and efficiency in scalable production environments for visually-rich document understanding (VDU) tasks. Cu…
The Common Stability Mechanism behind most Self-Supervised Learning Approaches
Abhishek Jha, Matthew B. Blaschko, Yuki M. Asano +1
Last couple of years have witnessed a tremendous progress in self-supervised learning (SSL), the success of which can be attributed to the introduction of useful inductive biases i…
Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errors
Teodora Popordanoska, Sebastian G. Gruber, Aleksei Tiulpin +2
Proper scoring rules evaluate the quality of probabilistic predictions, playing an essential role in the pursuit of accurate and well-calibrated models. Every proper score decompos…
Estimating calibration error under label shift without labels
Teodora Popordanoska, Gorjan Radevski, Tinne Tuytelaars +1
In the face of dataset shift, model calibration plays a pivotal role in ensuring the reliability of machine learning systems. Calibration error (CE) is an indicator of the alignmen…