KDSTM: Neural Semi-supervised Topic Modeling with Knowledge Distillation

Weijie Xu, Xiaoyu Jiang, Jay Desai +3

In text classification tasks, fine tuning pretrained language models like BERT and GPT-3 yields competitive accuracy; however, both methods require pretraining on large text datase…