3 papers
cs.AI2024
Mixture of Modular Experts: Distilling Knowledge from a Multilingual Teacher into Specialized Modular Language Models
Mohammed Al-Maamari, Mehdi Ben Amor, Michael Granitzer
This research combines Knowledge Distillation (KD) and Mixture of Experts (MoE) to develop modular, efficient multilingual language models. Key objectives include evaluating adapti…
cs.LG2023
Towards Measuring Representational Similarity of Large Language Models
Max Klabunde, Mehdi Ben Amor, Michael Granitzer +1
Understanding the similarity of the numerous released large language models (LLMs) has many uses, e.g., simplifying model selection, detecting illegal model reuse, and advancing ou…
cs.CL2023
Technical Report: Impact of Position Bias on Language Models in Token Classification
Mehdi Ben Amor, Michael Granitzer, Jelena Mitrović
Language Models (LMs) have shown state-of-the-art performance in Natural Language Processing (NLP) tasks. Downstream tasks such as Named Entity Recognition (NER) or Part-of-Speech…