5 papers
Active Continual Learning with Metaplastic Binary Bayesian Neural Networks
Kellian Cottart, Théo Ballet, Djohan Bonnet +1
Always-on edge systems must keep learning as conditions change under tight compute budgets and must detect unreliable predictions. Bayesian binary neural networks are attractive in…
Learning to Remember, Learn, and Forget in Attention-Based Models
Djohan Bonnet, Jamie Lohoff, Jan Finkbeiner +2
In-Context Learning (ICL) in transformers acts as an online associative memory and is believed to underpin their high performance on complex sequence processing tasks. However, in…
Forward-only learning in memristor arrays with month-scale stability
Adrien Renaudineau, Mamadou Hawa Diallo, Théo Dupuis +12
Turning memristor arrays from efficient inference engines into systems capable of on-chip learning has proved difficult. Weight updates have a high energy cost and cause device wea…
Bayesian continual learning and forgetting in neural networks
Djohan Bonnet, Kellian Cottart, Tifenn Hirtzlin +4
Biological synapses effortlessly balance memory retention and flexibility, yet artificial neural networks still struggle with the extremes of catastrophic forgetting and catastroph…
Bayesian Metaplasticity from Synaptic Uncertainty
Djohan Bonnet, Tifenn Hirtzlin, Tarcisius Januel +3
Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (ME…