6 papers
Lipschitz Continuity in Deep Learning: A Systematic Review of Theoretical Foundations, Estimation Methods, Regularization Approaches, and Certifiable Robustness
RóisÃn Luo, James McDermott, Colm O'Riordan
Lipschitz continuity is a fundamental property of neural networks that characterizes their sensitivity to input perturbations. It plays a pivotal role in deep learning, governing \…
Interpreting Global Perturbation Robustness of Image Models using Axiomatic Spectral Importance Decomposition
RóisÃn Luo, James McDermott, Colm O'Riordan
Perturbation robustness evaluates the vulnerabilities of models, arising from a variety of perturbations, such as data corruptions and adversarial attacks. Understanding the mechan…
Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization
RóisÃn Luo, Alexandru Drimbarean, James McDermott +1
This paper explores a novel paradigm in low-bit (i.e. 4-bits or lower) quantization, differing from existing state-of-the-art methods, by framing optimal quantization as an archite…
Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks
RóisÃn Luo, James McDermott, Christian Gagné +2
Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations; yet its dynamics (i.e. temporal evolution) during training remains un…
Higher-Order Singular-Value Derivatives of Rectangular Real Matrices
RóisÃn Luo, James McDermott, Colm O'Riordan
We present a theoretical framework for deriving the general -th order Fréchet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent opera…
Sampling Matters in Explanations: Towards Trustworthy Attribution Analysis Building Block in Visual Models through Maximizing Explanation Certainty
RóisÃn Luo, James McDermott, Colm O'Riordan
Image attribution analysis seeks to highlight the feature representations learned by visual models such that the highlighted feature maps can reflect the pixel-wise importance of i…