52 citations · 153 across the 67 of their papers we have counts for
72 papers
Understanding Stability of Outer Nesterov Momentum on Stochastic Pseudo-Gradients
Nishant Sharma, Laura Gomezjurado Gonzalez, Ganesh Talluri +2
Large neural networks are commonly trained across many workers, and communication between them is a major cost. Workers therefore take several local optimization steps before sendi…
Global Exponential Convergence of Two-Layer Linear Network Training
Stephen Y Zhang, Gabriel Peyré
We prove global exponential (linear) convergence with an explicit rate in the rich scaling for wide two-layer linear networks trained with smooth Polyak-Lojasiewicz predictor losse…
What Does Layer-Importance Reveal About Transformers and State-Space Models?
Istabrak Abbes, Nizar Islah, Irina Rish +1
Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers…
Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems
Aidan Li, Uday Kiran Reddy Tadipatri, Mahan Fathi +2
Koopman autoencoders (KAEs) seek a higher-dimensional latent representation in which nonlinear dynamics evolve linearly. However, many interesting systems have multiple basins of a…
Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
Maryam Hashemzadeh, Jerry Huang, Minseon Kim +2
The prevailing paradigm in large language model (LLM) alignment operates via erasure, filtering unsafe data or training models to strictly refuse harmful prompts. While effective a…
Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models
Nilaksh, Saurav Jha, Artem Zholus +1
World model-based policy evaluation is a practical proxy for testing real-world robot control by rolling out candidate actions in action-conditioned video diffusion models. As thes…
TAPNext++: What's Next for Tracking Any Point (TAP)?
Sebastian Jung, Artem Zholus, Martin Sundermeyer +6
Tracking-Any-Point (TAP) models aim to track any point through a video which is a crucial task in AR/XR and robotics applications. The recently introduced TAPNext approach proposes…
CoPeP: Benchmarking Continual Pretraining for Protein Language Models
Darshan Patil, Pranshu Malviya, Mathieu Reymond +2
Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, structure, and function from evolutionary sta…
The Expressive Limits of Diagonal SSMs for State-Tracking
Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh +1
State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-p…
LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents
Davide Baldelli, Ali Parviz, Amal Zouaq +1
As LLMs move from text completion toward autonomous agents, they remain constrained by the standard chat interface, which lacks private working memory. This raises a fundamental qu…
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Gabriele Prato, Shagun Sodhani, Alessandro Sordoni +1
The standard practice for training large language models involves packing multiple documents together to optimize computational efficiency. However, the impact of this process on t…
Neural Coherence : Find higher performance to out-of-distribution tasks from few samples
Simon Guiroy, Mats Richter, Sarath Chandar +1
To create state-of-the-art models for many downstream tasks, it has become common practice to fine-tune a pre-trained large vision model. However, it remains an open question of ho…
Shielded Controller Units for RL with Operational Constraints Applied to Remote Microgrids
Hadi Nekoei, Alexandre Blondin Massé, Rachid Hassani +2
Reinforcement learning (RL) is a powerful framework for optimizing decision-making in complex systems under uncertainty, an essential challenge in real-world settings, particularly…
The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
Milad Aghajohari, Kamran Chitsaz, Amirhossein Kazemnejad +4
Reinforcement learning (RL) has recently become a strong recipe for training reasoning LLMs that produce long chains of thought (LongCoT). Yet the standard RL "thinking environment…
JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation
Hadi Nekoei, Aman Jaiswal, Patrice Bechard +7
Large language model (LLM) agents perform well in sequential decision-making tasks, but improving them on unfamiliar domains often requires costly online interactions or fine-tunin…
GRPO-: Credit Assignment improves LLM Reasoning
Prasanna Parthasarathi, Mathieu Reymond, Boxing Chen +2
Large language models (LLMs) are increasingly deployed for tasks requiring complex reasoning, prompting significant interest in improving their reasoning abilities through post-tra…
CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learning
Prashant Govindarajan, Mathieu Reymond, Antoine Clavaud +3
In silico design and optimization of new materials primarily relies on high-accuracy atomic simulators that perform density functional theory (DFT) calculations. While recent works…
Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
Behnoush Khavari, Mehran Shakerinava, Jayesh Khullar +4
Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ran…
Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
Istabrak Abbes, Gopeshh Subbaraj, Matthew Riemer +6
Training large language models (LLMs) typically involves pre-training on massive corpora, only to restart the process entirely when new data becomes available. A more efficient and…
Optimizers Qualitatively Alter Solutions And We Should Leverage This
Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu +6
Due to the nonlinear nature of Deep Neural Networks (DNNs), one can not guarantee convergence to a unique global minimum of the loss when using optimizers relying only on local inf…
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Istabrak Abbes, Gabriele Prato, Quentin Fournier +4
Augmenting large language models (LLMs) with external context significantly improves their performance in natural language processing (NLP) tasks. However, LLMs struggle to answer…
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Mido Assran, Adrien Bardes, David Fan +27
A major challenge for modern AI is to learn to understand the world and learn to act largely by observation. This paper explores a self-supervised approach that combines internet-s…
Boosting LLM Reasoning via Spontaneous Self-Correction
Xutong Zhao, Tengyu Xu, Xuewei Wang +11
While large language models (LLMs) have demonstrated remarkable success on a broad range of tasks, math reasoning remains a challenging one. One of the approaches for improving mat…
NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment
Milan Bhan, Jean-Noel Vittaut, Nicolas Chesneau +2
Large Language Models (LLMs) can generate plausible free text self-explanations to justify their answers. However, these natural language explanations may not accurately reflect th…