176 citations · 826 across the 63 of their papers we have counts for
19 papers · 1 filter
Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray +1
This is a tutorial and survey paper on Generative Adversarial Network (GAN), adversarial autoencoders, and their variants. We start with explaining adversarial learning and the van…
Sufficient Dimension Reduction for High-Dimensional Regression and Low-Dimensional Embedding: Tutorial and Survey
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray +1
This is a tutorial and survey paper on various methods for Sufficient Dimension Reduction (SDR). We cover these methods with both statistical high-dimensional regression perspectiv…
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher
Mehdi Rezagholizadeh, Aref Jafari, Puneeth Salad +3
With ever growing scale of neural models, knowledge distillation (KD) attracts more attention as a prominent tool for neural model compression. However, there are counter intuitive…
KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Survey
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray +1
This is a tutorial and survey paper on Karush-Kuhn-Tucker (KKT) conditions, first-order and second-order numerical optimization, and distributed optimization. After a brief review…
Knowledge Distillation with Noisy Labels for Natural Language Understanding
Shivendra Bhardwaj, Abbas Ghaddar, Ahmad Rashid +5
Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications. However, one neglected area of…
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Tianda Li, Ahmad Rashid, Aref Jafari +3
Knowledge Distillation (KD) is a model compression algorithm that helps transfer the knowledge of a large neural network into a smaller one. Even though KD has shown promise on a w…