3 papers
cs.CV2024
Multi-Level Feature Distillation of Joint Teachers Trained on Distinct Image Datasets
Adrian Iordache, Bogdan Alexe, Radu Tudor Ionescu
We propose a novel teacher-student framework to distill knowledge from multiple teachers trained on distinct datasets. Each teacher is first trained from scratch on its own dataset…
cs.CV2023
Learning Diverse Features in Vision Transformers for Improved Generalization
Armand Mihai Nicolicioiu, Andrei Liviu Nicolicioiu, Bogdan Alexe +1
Deep learning models often rely only on a small set of features even when there is a rich set of predictive signals in the training data. This makes models brittle and sensitive to…
cs.CV2023
JEDI: Joint Expert Distillation in a Semi-Supervised Multi-Dataset Student-Teacher Scenario for Video Action Recognition
Lucian Bicsi, Bogdan Alexe, Radu Tudor Ionescu +1
We propose JEDI, a multi-dataset semi-supervised learning method, which efficiently combines knowledge from multiple experts, learned on different datasets, to train and improve th…