2 papers
cs.CV2021
Multi-granularity for knowledge distillation
Baitan Shao, Ying Chen
Considering the fact that students have different abilities to understand the knowledge imparted by teachers, a multi-granularity distillation mechanism is proposed for transferrin…
cs.CL2021
NT5?! Training T5 to Perform Numerical Reasoning
Peng-Jian Yang, Ying Ting Chen, Yuechan Chen +1
Numerical reasoning over text (NRoT) presents unique challenges that are not well addressed by existing pre-training objectives. We explore five sequential training schedules that…