63 citations · 235 across the 69 of their papers we have counts for
25 papers · 1 filter
Sharpness-Aware Minimization in Logit Space Efficiently Enhances Direct Preference Optimization
Haocheng Luo, Zehang Deng, Thanh-Toan Do +3
Direct Preference Optimization (DPO) has emerged as a popular algorithm for aligning pretrained large language models with human preferences, owing to its simplicity and training s…
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Quoc Minh Nguyen, Trung Le, Jing Wu +2
Fine-tuning-as-a-service introduces a threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, a process known as…
Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise
Haocheng Luo, Mehrtash Harandi, Dinh Phung +1
Sharpness-aware minimization (SAM) has emerged as a highly effective technique to improve model generalization, but its underlying principles are not fully understood. We investiga…
Curvature Learning for Generalization of Hyperbolic Neural Networks
Xiaomeng Fan, Yuwei Wu, Zhi Gao +2
Hyperbolic neural networks (HNNs) have demonstrated notable efficacy in representing real-world data with hierarchical structures via exploiting the geometric properties of hyperbo…
Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
Amara Zuffer, Michael Burke, Mehrtash Harandi
The diversity of tasks and dynamic nature of reinforcement learning (RL) require RL agents to be able to learn sequentially and continuously, a learning paradigm known as continuou…
Flashbacks to Harmonize Stability and Plasticity in Continual Learning
Leila Mahmoodi, Peyman Moghadam, Munawar Hayat +2
We introduce Flashback Learning (FL), a novel method designed to harmonize the stability and plasticity of models in Continual Learning (CL). Unlike prior approaches that primarily…
Exemplar-Free Continual Learning for State Space Models
Isaac Ning Lee, Leila Mahmoodi, Trung Le +1
State-Space Models (SSMs) excel at capturing long-range dependencies with structured recurrence, making them well-suited for sequence modeling. However, their evolving internal sta…
Optimizing Specific and Shared Parameters for Efficient Parameter Tuning
Van-Anh Nguyen, Thanh-Toan Do, Mehrtash Harandi +2
Foundation models, with a vast number of parameters and pretraining on massive datasets, achieve state-of-the-art performance across various applications. However, efficiently adap…
Why Domain Generalization Fail? A View of Necessity and Sufficiency
Long-Tung Vuong, Vy Vo, Hien Dang +5
Despite a strong theoretical foundation, empirical experiments reveal that existing domain generalization (DG) algorithms often fail to consistently outperform the ERM baseline. We…
Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization
Haocheng Luo, Tuan Truong, Tung Pham +3
Sharpness-Aware Minimization (SAM) has attracted significant attention for its effectiveness in improving generalization across various tasks. However, its underlying principles re…
Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks
Jing Wu, Mehrtash Harandi
Machine unlearning has become a pivotal task to erase the influence of data from a trained model. It adheres to recent data regulation standards and enhances the privacy and securi…
Sharpness-Aware Teleportation on Riemannian Manifolds
Tuan Truong, Hoang-Phi Nguyen, Haocheng Luo +4
Recent studies highlight the effectiveness of flat minima in enhancing generalization, with sharpness-aware minimization (SAM) achieving state-of-the-art performance. Additionally,…
L3DMC: Lifelong Learning using Distillation via Mixed-Curvature Space
Kaushik Roy, Peyman Moghadam, Mehrtash Harandi
The performance of a lifelong learning (L3) model degrades when it is trained on a series of tasks, as the geometrical formation of the embedding space changes while learning novel…
Vector Quantized Wasserstein Auto-Encoder
Tung-Long Vuong, Trung Le, He Zhao +4
Learning deep discrete latent presentations offers a promise of better symbolic and summarized abstractions that are more useful to subsequent downstream tasks. Inspired by the sem…
Concealing Sensitive Samples against Gradient Leakage in Federated Learning
Jing Wu, Munawar Hayat, Mingyi Zhou +1
Federated Learning (FL) is a distributed learning paradigm that enhances users privacy by eliminating the need for clients to share raw, private data with the server. Despite the s…
On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptation
Markus Hiller, Mehrtash Harandi, Tom Drummond
Inspired by the concept of preconditioning, we propose a novel method to increase adaptation speed for gradient-based meta-learning methods without incurring extra parameters. We d…
On Generalizing Beyond Domains in Cross-Domain Continual Learning
Christian Simon, Masoud Faraki, Yi-Hsuan Tsai +5
Humans have the ability to accumulate knowledge of new tasks in varying conditions, but deep neural networks often suffer from catastrophic forgetting of previously learned knowled…
Towards a Robust Differentiable Architecture Search under Label Noise
Christian Simon, Piotr Koniusz, Lars Petersson +2
Neural Architecture Search (NAS) is the game changer in designing robust neural architectures. Architectures designed by NAS outperform or compete with the best manual network desi…
Dense Uncertainty Estimation
Jing Zhang, Yuchao Dai, Mochu Xiang +7
Deep neural networks can be roughly divided into deterministic neural networks and stochastic neural networks.The former is usually trained to achieve a mapping from input space to…
Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
Jieming Zhou, Tong Zhang, Pengfei Fang +2
Prior to the introduction of Graph Neural Networks (GNNs), modeling and analyzing irregular data, particularly graphs, was thought to be the Achilles' heel of deep learning. The co…
Semi-Supervised Metric Learning: A Deep Resurrection
Ujjal Kr Dutta, Mehrtash Harandi, Chellu Chandra Sekhar
Distance Metric Learning (DML) seeks to learn a discriminative embedding where similar examples are closer, and dissimilar examples are apart. In this paper, we address the problem…
Performance Evaluation of Adversarial Attacks: Discrepancies and Solutions
Jing Wu, Mingyi Zhou, Ce Zhu +3
Recently, adversarial attack methods have been developed to challenge the robustness of machine learning models. However, mainstream evaluation criteria experience limitations, eve…
On Learning the Geodesic Path for Incremental Learning
Christian Simon, Piotr Koniusz, Mehrtash Harandi
Neural networks notoriously suffer from the problem of catastrophic forgetting, the phenomenon of forgetting the past knowledge when acquiring new knowledge. Overcoming catastrophi…
Learning Log-Determinant Divergences for Positive Definite Matrices
Anoop Cherian, Panagiotis Stanitsas, Jue Wang +3
Representations in the form of Symmetric Positive Definite (SPD) matrices have been popularized in a variety of visual learning applications due to their demonstrated ability to ca…