4 papers
A Survey of Neural Trees
Haoling Li, Jie Song, Mengqi Xue +4
Neural networks (NNs) and decision trees (DTs) are both popular models of machine learning, yet coming with mutually exclusive advantages and limitations. To bring the best of the…
Meta-attention for ViT-backed Continual Learning
Mengqi Xue, Haofei Zhang, Jie Song +1
Continual learning is a longstanding research topic due to its crucial role in tackling continually arriving tasks. Up to now, the study of continual learning in computer vision is…
KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation
Mengqi Xue, Jie Song, Xinchao Wang +3
Knowledge distillation (KD) has recently emerged as an efficacious scheme for learning compact deep neural networks (DNNs). Despite the promising results achieved, the rationale th…
Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation
Chengchao Shen, Mengqi Xue, Xinchao Wang +3
A massive number of well-trained deep networks have been released by developers online. These networks may focus on different tasks and in many cases are optimized for different da…