7 papers
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
Michael Rizvi-Martel, Satwik Bhattamishra, Guillaume Rabusseau +1
A theoretical understanding of Transformers is crucial to better understand the capacities and limitations of large language models (LLMs). There is much work analyzing the express…
On the Role of Depth in the Expressivity of RNNs
Maude Lizaire, Michael Rizvi-Martel, Éric Dupuis +1
The benefits of depth in feedforward neural networks are well known: composing multiple layers of linear transformations with nonlinear activations enables complex computations. Wh…
The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models
Michael Rizvi-Martel, Guillaume Rabusseau, Marius Mosbach
Latent reasoning via continuous chain-of-thoughts (Latent CoT) has emerged as a promising alternative to discrete CoT reasoning. Operating in continuous space increases expressivit…
FlowQ-Net: A Generative Framework for Automated Quantum Circuit Design
Jun Dai, Michael Rizvi-Martel, Guillaume Rabusseau
Designing efficient quantum circuits is a central bottleneck to exploring the potential of quantum computing, particularly for noisy intermediate-scale quantum (NISQ) devices, wher…
Benefits and Limitations of Communication in Multi-Agent Reasoning
Michael Rizvi-Martel, Satwik Bhattamishra, Neil Rathi +2
Chain-of-thought prompting has popularized step-by-step reasoning in large language models, yet model performance still degrades as problem complexity and context length grow. By d…
Numerical PDE solvers outperform neural PDE solvers
Patrick Chatain, Michael Rizvi-Martel, Guillaume Rabusseau +1
We present DeepFDM, a differentiable finite-difference framework for learning spatially varying coefficients in time-dependent partial differential equations (PDEs). By embedding a…
A Tensor Decomposition Perspective on Second-order RNNs
Maude Lizaire, Michael Rizvi-Martel, Marawan Gamal Abdel Hameed +1
Second-order Recurrent Neural Networks (2RNNs) extend RNNs by leveraging second-order interactions for sequence modelling. These models are provably more expressive than their firs…