1 citations · 2 across the 5 of their papers we have counts for
6 papers
A Unified Revisit of Temperature in Classification-Based Knowledge Distillation
Logan Frank, Jim Davis
A central idea of knowledge distillation is to expose relational structure embedded in the teacher's weights for the student to learn, which is often facilitated using a temperatur…
Assessing the Potential for Catastrophic Failure in Dynamic Post-Training Quantization
Logan Frank, Paul Ardis
Post-training quantization (PTQ) has recently emerged as an effective tool for reducing the computational complexity and memory usage of a neural network by representing its weight…
What Makes a Good Dataset for Knowledge Distillation?
Logan Frank, Jim Davis
Knowledge distillation (KD) has been a popular and effective method for model compression. One important assumption of KD is that the teacher's original dataset will also be availa…
Deep Learning Improvements for Sparse Spatial Field Reconstruction
Robert Sunderhaft, Logan Frank, Jim Davis
Accurately reconstructing a global spatial field from sparse data has been a longstanding problem in several domains, such as Earth Sciences and Fluid Dynamics. Historically, scien…
Data-Free Knowledge Distillation Using Adversarially Perturbed OpenGL Shader Images
Logan Frank, Jim Davis
Knowledge distillation (KD) has been a popular and effective method for model compression. One important assumption of KD is that the original training dataset is always available.…
Revisiting Batch Norm Initialization
Jim Davis, Logan Frank
Batch normalization (BN) is comprised of a normalization component followed by an affine transformation and has become essential for training deep neural networks. Standard initial…