5 papers
From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertise
Nitin Sharma, Thomas Wolfers, ÃaÄatay Yıldız
Accurate domain-specific benchmarking of LLMs is essential, specifically in domains with direct implications for humans, such as law, healthcare, and education. However, existing b…
Object-level Self-Distillation for Vision Pretraining
ÃaÄlar Hızlı, ÃaÄatay Yıldız, Pekka Marttinen
State-of-the-art vision pretraining methods rely on image-level self-distillation from object-centric datasets such as ImageNet, implicitly assuming each image contains a single ob…
Investigating Continual Pretraining in Large Language Models: Insights and Implications
ÃaÄatay Yıldız, Nishaanth Kanna Ravichandran, Nitin Sharma +2
Continual learning (CL) in large language models (LLMs) is an evolving domain that focuses on developing efficient and sustainable training strategies to adapt models to emerging k…
Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
Sebastian Dziadzio, ÃaÄatay Yıldız, Gido M. van de Ven +3
The ability of machine learning systems to learn continually is hindered by catastrophic forgetting, the tendency of neural networks to overwrite previously acquired knowledge when…
Identifying latent state transition in non-linear dynamical systems
ÃaÄlar Hızlı, ÃaÄatay Yıldız, Matthias Bethge +2
This work aims to improve generalization and interpretability of dynamical systems by recovering the underlying lower-dimensional latent states and their time evolutions. Previous…