6 papers
Fast and Slow Variational Continual Learning
Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan +3
Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural m…
CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models
Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey +4
Large language models (LLMs) excel at operating at scale by leveraging social media and various data crawled from the web. Whereas existing corpora are diverse, their frequent lack…
BOWL: A Deceptively Simple Open World Learner
Roshni . R. Kamath, Rupert Mitchell, Subarnaduti Paul +2
Traditional machine learning excels on static benchmarks, but the real world is dynamic and seldom as carefully curated as test sets. Practical applications may generally encounter…
Continual Learning Should Move Beyond Incremental Classification
Rupert Mitchell, Antonio Alliegro, Raffaello Camoriano +17
Continual learning (CL) is the sub-field of machine learning concerned with accumulating knowledge in dynamic environments. So far, CL research has mainly focused on incremental cl…
The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation
Martin Mundt, Anaelia Ovalle, Felix Friedrich +5
In a widely popular analogy by Turing Award Laureate Yann LeCun, machine intelligence has been compared to cake - where unsupervised learning forms the base, supervised learning ad…
Core Tokensets for Data-efficient Sequential Training of Transformers
Subarnaduti Paul, Manuel Brack, Patrick Schramowski +2
Deep networks are frequently tuned to novel tasks and continue learning from ongoing data streams. Such sequential training requires consolidation of new and past information, a ch…