2 citations · 2 across the 4 of their papers we have counts for
6 papers
On Reparameterizing the Score Function
Mohammad Hossein Amani, Leonardo Bocchieri, Maxime Peyrard +2
Policy gradients for autoregressive sequence models differentiate each token's log-probability while holding the conditioning history fixed. This discards information about how ear…
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
RL for Reasoning by Adaptively Revealing Rationales
Mohammad Hossein Amani, Aryo Lotfi, Nicolas Mario Baldwin +4
Learning in the combinatorially large output space of sequence generation problems is challenging as providing expert demonstrations scales poorly with sequence length, and RL stru…
Symbolic Autoencoding for Self-Supervised Sequence Learning
Mohammad Hossein Amani, Nicolas Mario Baldwin, Amin Mansouri +3
Traditional language models, adept at next-token prediction in text sequences, often struggle with transduction tasks between distinct symbolic systems, particularly when parallel…
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization
Simone Bombari, Mohammad Hossein Amani, Marco Mondelli
The Neural Tangent Kernel (NTK) has emerged as a powerful tool to provide memorization, optimization and generalization guarantees in deep neural networks. A line of work has studi…
Sharp asymptotics on the compression of two-layer neural networks
Mohammad Hossein Amani, Simone Bombari, Marco Mondelli +2
In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes. More precisely, we consider the setting in which…