248 citations · 710 across the 48 of their papers we have counts for
32 papers · 1 filter
Expert Merging in Sparse Mixture of Experts with Nash Bargaining
Dung V. Nguyen, Anh T. Nguyen, Minh H. Nguyen +6
Existing expert merging strategies for Sparse Mixture of Experts (SMoE) typically rely on input-dependent or input-independent averaging of expert parameters, but often lack a prin…
Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalization
Aviv Shamsian, Eitan Shaar, Aviv Navon +2
Machine unlearning aims to remove the influence of problematic training data after a model has been trained. The primary challenge in machine unlearning is ensuring that the proces…
Adversarial Attacks in Weight-Space Classifiers
Tamir Shor, Ethan Fetaya, Chaim Baskin +1
Implicit Neural Representations (INRs) have been recently garnering increasing interest in various research fields, mainly due to their ability to represent large, complex data in…
Conformal Prediction of Classifiers with Many Classes based on Noisy Labels
Coby Penso, Jacob Goldberger, Ethan Fetaya
Conformal Prediction (CP) controls the prediction uncertainty of classification systems by producing a small prediction set, ensuring a predetermined probability that the true clas…
Multi Task Inverse Reinforcement Learning for Common Sense Reward
Neta Glazer, Aviv Navon, Aviv Shamsian +1
One of the challenges in applying reinforcement learning in a complex real-world environment lies in providing the agent with a sufficiently detailed reward function. Any misalignm…
Improved Generalization of Weight Space Networks via Augmentations
Aviv Shamsian, Aviv Navon, David W. Zhang +4
Learning in deep weight spaces (DWS), where neural networks process the weights of other neural networks, is an emerging research direction, with applications to 2D and 3D neural f…
Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning
Idan Achituve, Idit Diamant, Arnon Netzer +2
As machine learning becomes more prominent there is a growing demand to perform several inference tasks in parallel. Running a dedicated model for each task is computationally expe…
Data Augmentations in Deep Weight Spaces
Aviv Shamsian, David W. Zhang, Aviv Navon +10
Learning in weight spaces, where neural networks process the weights of other deep neural networks, has emerged as a promising research direction with applications in various field…
Equivariant Deep Weight Space Alignment
Aviv Navon, Aviv Shamsian, Ethan Fetaya +3
Permutation symmetries of deep networks make basic operations like model merging and similarity estimation challenging. In many cases, aligning the weights of the networks, i.e., f…
Learning Discrete Weights and Activations Using the Local Reparameterization Trick
Guy Berger, Aviv Navon, Ethan Fetaya
In computer vision and machine learning, a crucial challenge is to lower the computation and memory demands for neural network inference. A commonplace solution to address this cha…
Guided Deep Kernel Learning
Idan Achituve, Gal Chechik, Ethan Fetaya
Combining Gaussian processes with the expressive power of deep neural networks is commonly done nowadays through deep kernel learning (DKL). Unfortunately, due to the kernel optimi…
Auxiliary Learning as an Asymmetric Bargaining Game
Aviv Shamsian, Aviv Navon, Neta Glazer +3
Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may…
Equivariant Architectures for Learning in Deep Weight Spaces
Aviv Navon, Aviv Shamsian, Idan Achituve +3
Designing machine learning architectures for processing neural networks in their raw weight matrix form is a newly introduced research direction. Unfortunately, the unique symmetry…
A Study on the Evaluation of Generative Models
Eyal Betzalel, Coby Penso, Aviv Navon +1
Implicit generative models, which do not return likelihood values, such as generative adversarial networks and diffusion models, have become prevalent in recent years. While it is…
Functional Ensemble Distillation
Coby Penso, Idan Achituve, Ethan Fetaya
Bayesian models have many desirable properties, most notable is their ability to generalize from limited data and to properly estimate the uncertainty in their predictions. However…
Multi-Task Learning as a Bargaining Game
Aviv Navon, Aviv Shamsian, Idan Achituve +4
In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency;…
Personalized Federated Learning with Gaussian Processes
Idan Achituve, Aviv Shamsian, Aviv Navon +2
Federated learning aims to learn a global model that performs well on client devices with limited cross-client communication. Personalized federated learning (PFL) further extends…
Personalized Federated Learning using Hypernetworks
Aviv Shamsian, Aviv Navon, Ethan Fetaya +1
Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models…
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
Idan Achituve, Aviv Navon, Yochai Yemini +2
Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially co…
From Local Structures to Size Generalization in Graph Neural Networks
Gilad Yehudai, Ethan Fetaya, Eli Meirom +2
Graph neural networks (GNNs) can process graphs of different sizes, but their ability to generalize across sizes, specifically from small to large graphs, is still not well underst…
Learning the Pareto Front with Hypernetworks
Aviv Navon, Aviv Shamsian, Gal Chechik +1
Multi-objective optimization (MOO) problems are prevalent in machine learning. These problems have a set of optimal solutions, called the Pareto front, where each point on the fron…
On Learning Sets of Symmetric Elements
Haggai Maron, Or Litany, Gal Chechik +1
Learning from unordered sets is a fundamental learning setup, recently attracting increasing attention. Research in this area has focused on the case where elements of the set are…
Understanding the Limitations of Conditional Generative Models
Ethan Fetaya, Jörn-Henrik Jacobsen, Will Grathwohl +1
Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts. They are a natural choice to solve discriminative tasks in a rob…
Evaluating and Calibrating Uncertainty Prediction in Regression Tasks
Dan Levi, Liran Gispan, Niv Giladi +1
Predicting not only the target but also an accurate measure of uncertainty is important for many machine learning applications and in particular safety-critical ones. In this work…