4 papers
LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
Hossein Rajabzadeh, Maryam Dialameh, Chul B. Park +2
Autoregressive large language models (LLMs) are bottlenecked by sequential decoding, where each new token typically requires executing all transformer layers. Existing dynamic-dept…
Autoencoder-Based Hybrid Replay for Class-Incremental Learning
Milad Khademi Nori, Il-Min Kim, Guanghui Wang
In class-incremental learning (CIL), effective incremental learning strategies are essential to mitigate task confusion and catastrophic forgetting, especially as the number of tas…
Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting
Milad Khademi Nori, Il-Min Kim, Guanghui Wang
Federated Class-Incremental Learning (FCIL) refers to a scenario where a dynamically changing number of clients collaboratively learn an ever-increasing number of incoming tasks. F…
Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings
Milad Khademi Nori, Il-Min Kim
In class-incremental learning (class-IL), models must classify all previously seen classes at test time without task-IDs, leading to task confusion. Despite being a key challenge,…