6 papers
Lipschitz Continuity in Deep Learning: A Systematic Review of Theoretical Foundations, Estimation Methods, Regularization Approaches, and Certifiable Robustness
RóisÃn Luo, James McDermott, Colm O'Riordan
Lipschitz continuity is a fundamental property of neural networks that characterizes their sensitivity to input perturbations. It plays a pivotal role in deep learning, governing \…
Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization
RóisÃn Luo, Alexandru Drimbarean, James McDermott +1
This paper explores a novel paradigm in low-bit (i.e. 4-bits or lower) quantization, differing from existing state-of-the-art methods, by framing optimal quantization as an archite…
Can Synthetic Data Improve Symbolic Regression Extrapolation Performance?
Fitria Wulandari Ramlan, Colm O'Riordan, Gabriel Kronberger +1
Many machine learning models perform well when making predictions within the training data range, but often struggle when required to extrapolate beyond it. Symbolic regression (SR…
Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks
RóisÃn Luo, James McDermott, Christian Gagné +2
Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations; yet its dynamics (i.e. temporal evolution) during training remains un…
Higher-Order Singular-Value Derivatives of Rectangular Real Matrices
RóisÃn Luo, James McDermott, Colm O'Riordan
We present a theoretical framework for deriving the general -th order Fréchet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent opera…
Sampling Matters in Explanations: Towards Trustworthy Attribution Analysis Building Block in Visual Models through Maximizing Explanation Certainty
RóisÃn Luo, James McDermott, Colm O'Riordan
Image attribution analysis seeks to highlight the feature representations learned by visual models such that the highlighted feature maps can reflect the pixel-wise importance of i…