6 citations · 12 across the 8 of their papers we have counts for
13 papers
IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leaves
Feyza Yavuz, Mert Bülent Sarıyıldız, Diane Larlus
Multi-teacher distillation has emerged as a way to combine complementary teacher models into a single student model that exhibits the strengths of all its teachers. The student is…
Compressing Observation History into Agent Memory: Distilling Transformers into Recurrent Transformers
Philippe Weinzaepfel, Christian Wolf, Bülent Mert Sariyildiz +2
Transformers are AI's workhorse with strong performance in modeling sequential data, but their computational cost becomes prohibitive when processing long sequences. We target long…
Task Alignment: A Simple Proxy for Practical Model Merging Across Diverse Vision Tasks
Pau de Jorge, César Roberto de Souza, Björn Michele +5
Efficiently merging several models fine-tuned for different tasks, but stemming from the same pretrained base model, is of great practical interest. Despite extensive prior work, m…
Kinaema: a recurrent sequence model for memory and pose in motion
Mert Bulent Sariyildiz, Philippe Weinzaepfel, Guillaume Bono +2
One key aspect of spatially aware robots is the ability to "find their bearings", ie. to correctly situate themselves in previously seen spaces. In this work, we focus on this part…
DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers
Mert Bulent Sariyildiz, Philippe Weinzaepfel, Thomas Lucas +3
Recent multi-teacher distillation methods have unified the encoders of multiple foundation models into a single encoder, achieving competitive performance on core vision tasks like…
UNIC: Universal Classification Models via Multi-teacher Distillation
Mert Bulent Sariyildiz, Philippe Weinzaepfel, Thomas Lucas +2
Pretrained models have become a commodity and offer strong results on a broad range of tasks. In this work, we focus on classification and seek to learn a unique encoder able to ta…