6 papers
Entropic Riemannian Neural Optimal Transport
Alessandro Micheli, Silvia Sapora, Anthea Monod +1
Many machine learning problems involve data supported on curved spaces such as spheres, rotation groups, hyperbolic spaces, and general Riemannian manifolds, where Euclidean geomet…
Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
Jonathan Cook, Silvia Sapora, Arash Ahmadian +4
Large language models (LLMs) are typically trained to acquire behaviours from demonstrations or experience, yet much of their training data is declarative: instructions, rules, and…
GRACE: A Language Model Framework for Explainable Inverse Reinforcement Learning
Silvia Sapora, Devon Hjelm, Alexander Toshev +2
Inverse Reinforcement Learning aims to recover reward models from expert demonstrations, but traditional methods yield black-box models that are difficult to interpret and debug. I…
Meta-Learning Objectives for Preference Optimization
Carlo Alfano, Silvia Sapora, Jakob Nicolaus Foerster +2
Evaluating preference optimization (PO) algorithms on LLM alignment is a challenging task that presents prohibitive costs, noise, and several variables like model size and hyper-pa…
Learning mirror maps in policy mirror descent
Carlo Alfano, Sebastian Towers, Silvia Sapora +2
Policy Mirror Descent (PMD) is a popular framework in reinforcement learning, serving as a unifying perspective that encompasses numerous algorithms. These algorithms are derived t…
The Complexity Dynamics of Grokking
Branton DeMoss, Silvia Sapora, Jakob Foerster +2
We demonstrate the existence of a complexity phase transition in neural networks by studying the grokking phenomenon, where networks suddenly transition from memorization to genera…