312 citations · 676 across the 20 of their papers we have counts for
23 papers
Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions
Ahmed El Kady, Aravind Narayanan, Rehana Riaz +2
Efficient evaluation changes the protocol used to support claims about model behavior, yet it is rarely tested whether those claims remain stable after the evaluation itself is mad…
SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
Mohammed Adnan, Rohan Jain, Tom Jacobs +4
Dynamic Sparse Training (DST) methods train neural networks by maintaining sparsity while dynamically adapting the network topology. Despite the promise of reduced computation, DST…
Reducing Hallucinations in LLMs via Factuality-Aware Preference Learning
Sindhuja Chaduvula, Ahmed Y. Radwan, Azib Farooq +2
Preference alignment methods such as RLHF and Direct Preference Optimization (DPO) improve instruction following, but they can also reinforce hallucinations when preference judgmen…
REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
Mike Lasby, Ivan Lazarevich, Nish Sinnadurai +3
Sparsely-activated Mixture-of-Experts (SMoE) models offer efficient pre-training and low latency but their large parameter counts create significant memory overhead, motivating res…
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry
Mohammed Adnan, Rohan Jain, Ekansh Sharma +2
The Lottery Ticket Hypothesis (LTH) suggests there exists a sparse LTH mask and weights that achieve the same generalization performance as the dense model while using significantl…
Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition
Cem Üyük, Mike Lasby, Mohamed Yassin +2
Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices. Among existing compression approa…
Navigating Extremes: Dynamic Sparsity in Large Output Spaces
Nasib Ullah, Erik Schultheis, Mike Lasby +2
In recent years, Dynamic Sparse Training (DST) has emerged as an alternative to post-training pruning for generating efficient models. In principle, DST allows for a more memory ef…
What is Left After Distillation? How Knowledge Transfer Impacts Fairness and Bias
Aida Mohammadshahi, Yani Ioannou
Knowledge Distillation is a commonly used Deep Neural Network (DNN) compression method, which often maintains overall generalization performance. However, we show that even for bal…
Trustworthy and Responsible AI for Human-Centric Autonomous Decision-Making Systems
Farzaneh Dehghani, Mahsa Dibaji, Fahim Anzum +23
Artificial Intelligence (AI) has paved the way for revolutionary decision-making processes, which if harnessed appropriately, can contribute to advancements in various sectors, fro…
Meta-GCN: A Dynamically Weighted Loss Minimization Method for Dealing with the Data Imbalance in Graph Neural Networks
Mahdi Mohammadizadeh, Arash Mozhdehi, Yani Ioannou +1
Although many real-world applications, such as disease prediction, and fault detection suffer from class imbalance, most existing graph-based classification methods ignore the skew…
Dynamic Sparse Training with Structured Sparsity
Mike Lasby, Anna Golubeva, Utku Evci +2
Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training…
Bounding generalization error with input compression: An empirical study with infinite-width networks
Angus Galloway, Anna Golubeva, Mahmoud Salem +3
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE ba…
Monitoring Shortcut Learning using Mutual Information
Mohammed Adnan, Yani Ioannou, Chuan-Yung Tsai +3
The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critica…
Domain-Agnostic Clustering with Self-Distillation
Mohammed Adnan, Yani A. Ioannou, Chuan-Yung Tsai +1
Recent advancements in self-supervised learning have reduced the gap between supervised and unsupervised representation learning. However, most self-supervised and deep clustering…
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
Utku Evci, Yani A. Ioannou, Cem Keskin +1
Sparse Neural Networks (NNs) can match the generalization of dense NNs using a fraction of the compute/storage for inference, and also have the potential to enable efficient traini…
Rapid Classification of TESS Planet Candidates with Convolutional Neural Networks
Hugh P. Osborn, Megan Ansdell, Yani Ioannou +6
Accurately and rapidly classifying exoplanet candidates from transit surveys is a goal of growing importance as the data rates from space-based survey missions increases. This is e…
Scientific Domain Knowledge Improves Exoplanet Transit Classification with Deep Learning
Megan Ansdell, Yani Ioannou, Hugh P. Osborn +5
Space-based missions such as Kepler, and soon TESS, provide large datasets that must be analyzed efficiently and systematically. Recent work by Shallue & Vanderburg (2018) successf…
Measuring Neural Net Robustness with Constraints
Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos +3
Despite having high accuracy, neural nets have been shown to be susceptible to adversarial examples, where a small perturbation to an input can cause it to become mislabeled. We pr…
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Yani Ioannou, Duncan Robertson, Roberto Cipolla +1
We propose a new method for creating computationally efficient and compact convolutional neural networks (CNNs) using a novel sparse connection structure that resembles a tree root…
Refining Architectures of Deep Convolutional Neural Networks
Sukrit Shankar, Duncan Robertson, Yani Ioannou +2
Deep Convolutional Neural Networks (CNNs) have recently evinced immense success for various image recognition tasks. However, a question of paramount importance is somewhat unanswe…
Decision Forests, Convolutional Networks and the Models in-Between
Yani Ioannou, Duncan Robertson, Darko Zikic +4
This paper investigates the connections between two state of the art classifiers: decision forests (DFs, including decision jungles) and convolutional neural networks (CNNs). Decis…
Training CNNs with Low-Rank Filters for Efficient Image Classification
Yani Ioannou, Duncan Robertson, Jamie Shotton +2
We propose a new method for creating computationally efficient convolutional neural networks (CNNs) by using low-rank representations of convolutional filters. Rather than approxim…
Difference of Normals as a Multi-Scale Operator in Unorganized Point Clouds
Yani Ioannou, Babak Taati, Robin Harrap +1
A novel multi-scale operator for unorganized 3D point clouds is introduced. The Difference of Normals (DoN) provides a computationally efficient, multi-scale approach to processing…