77 citations · 732 across the 55 of their papers we have counts for
10 papers · 1 filter
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization
Aref Jafari, Ivan Kobyzev, Mehdi Rezagholizadeh +2
Knowledge Distillation (KD) has been extensively used for natural language understanding (NLU) tasks to improve a small model's (a student) generalization by transferring the knowl…
Label Alignment Regularization for Distribution Shift
Ehsan Imani, Guojun Zhang, Runjia Li +4
Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular v…
Learning Functions on Multiple Sets using Multi-Set Transformers
Kira Selby, Ahmad Rashid, Ivan Kobyzev +2
We propose a general deep architecture for learning functions on multiple permutation-invariant sets. We also show how to generalize this architecture to sets of elements of any di…
Robust One Round Federated Learning with Predictive Space Bayesian Inference
Mohsin Hasan, Zehao Zhang, Kaiyang Guo +4
Making predictions robust is an important challenge. A separate challenge in federated learning (FL) is to reduce the number of communication rounds, particularly since doing so re…
Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process
Haolin Yu, Kaiyang Guo, Mahdi Karami +3
In typical scenarios where the Federated Learning (FL) framework applies, it is common for clients to have insufficient training data to produce an accurate model. Thus, models tha…
Benchmarking Constraint Inference in Inverse Reinforcement Learning
Guiliang Liu, Yudong Luo, Ashish Gaurav +2
When deploying Reinforcement Learning (RL) agents into a physical system, we must ensure that these agents are well aware of the underlying constraints. In many real-world problems…