56 citations · 170 across the 32 of their papers we have counts for
33 papers
Grokking Finite-Dimensional Algebra
Pascal Jr Tikeng Notsawo, Guillaume Dumas, Guillaume Rabusseau
This paper investigates the grokking phenomenon, which refers to the sudden transition from a long memorization to generalization observed during neural networks training, in the c…
ClustRecNet: A Novel End-to-End Deep Learning Framework for Clustering Algorithm Recommendation
Mohammadreza Bakhtyari, Bogdan Mazoure, Renato Cordeiro de Amorim +2
Identifying an effective clustering algorithm for a given dataset remains a fundamental unsupervised learning issue. We introduce ClustRecNet, a novel end-to-end deep learning fram…
Simulating Weighted Automata over Sequences and Trees with Transformers
Michael Rizvi, Maude Lizaire, Clara Lacroce +1
Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the past few years. However, little is unders…
Generative Learning of Continuous Data by Tensor Networks
Alex Meiburg, Jing Chen, Jacob Miller +3
Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervise…
Optimal Approximate Minimization of One-Letter Weighted Finite Automata
Clara Lacroce, Borja Balle, Prakash Panangaden +1
In this paper, we study the approximate minimization problem of weighted finite automata (WFAs): to compute the best possible approximation of a WFA given a bound on the number of…
Spectral Regularization: an Inductive Bias for Sequence Modeling
Kaiwen Hou, Guillaume Rabusseau
Various forms of regularization in learning tasks strive for different notions of simplicity. This paper presents a spectral regularization technique, which attaches a unique induc…
Towards an AAK Theory Approach to Approximate Minimization in the Multi-Letter Case
Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau
We study the approximate minimization problem of weighted finite automata (WFAs): given a WFA, we want to compute its optimal approximation when restricted to a given size. We refo…
Extracting Weighted Automata for Approximate Minimization in Language Modelling
Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau
In this paper we study the approximate minimization problem for language modelling. We assume we are given some language model as a black box. The objective is to obtain a weighted…
Lower and Upper Bounds on the VC-Dimension of Tensor Network Models
Behnoush Khavari, Guillaume Rabusseau
Tensor network methods have been a key ingredient of advances in condensed matter physics and have recently sparked interest in the machine learning community for their ability to…
Optimal Spectral-Norm Approximate Minimization of Weighted Finite Automata
Borja Balle, Clara Lacroce, Prakash Panangaden +2
We address the approximate minimization problem for weighted finite automata (WFAs) with weights in , over a one-letter alphabet: to compute the best possible approxima…
Assessing the Impact: Does an Improvement to a Revenue Management System Lead to an Improved Revenue?
Greta Laage, Emma Frejinger, Andrea Lodi +1
Airlines and other industries have been making use of sophisticated Revenue Management Systems to maximize revenue for decades. While improving the different components of these sy…
Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
Siddarth Srinivasan, Sandesh Adhikary, Jacob Miller +2
Modeling joint probability distributions over sequences has been studied from many perspectives. The physics community developed matrix product states, a tensor-train decomposition…
Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning
Tianyu Li, Doina Precup, Guillaume Rabusseau
In this paper, we present connections between three models used in different research fields: weighted finite automata~(WFA) from formal languages and linguistics, recurrent neural…
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Thang Doan, Mehdi Bennani, Bogdan Mazoure +2
Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although major advances have been made in the field,…
Adaptive Learning of Tensor Network Structures
Meraj Hashemizadeh, Michelle Liu, Jacob Miller +1
Tensor Networks (TN) offer a powerful framework to efficiently represent very high-dimensional objects. TN have recently shown their potential for machine learning applications and…
Laplacian Change Point Detection for Dynamic Graphs
Shenyang Huang, Yasmeen Hitti, Guillaume Rabusseau +1
Dynamic and temporal graphs are rich data structures that are used to model complex relationships between entities over time. In particular, anomaly detection in temporal graphs is…
Tensorized Random Projections
Beheshteh T. Rakhshan, Guillaume Rabusseau
We introduce a novel random projection technique for efficiently reducing the dimension of very high-dimensional tensors. Building upon classical results on Gaussian random project…
RandomNet: Towards Fully Automatic Neural Architecture Design for Multimodal Learning
Stefano Alletto, Shenyang Huang, Vincent Francois-Lavet +2
Almost all neural architecture search methods are evaluated in terms of performance (i.e. test accuracy) of the model structures that it finds. Should it be the only metric for a g…
Tensor Networks for Probabilistic Sequence Modeling
Jacob Miller, Guillaume Rabusseau, John Terilla
Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we util…
Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces
Bogdan Mazoure, Thang Doan, Tianyu Li +4
We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional embedding of the policy on a reproducin…
Efficient Planning under Partial Observability with Unnormalized Q Functions and Spectral Learning
Tianyu Li, Bogdan Mazoure, Doina Precup +1
Learning and planning in partially-observable domains is one of the most difficult problems in reinforcement learning. Traditional methods consider these two problems as independen…
Neural Architecture Search for Class-incremental Learning
Shenyang Huang, Vincent François-Lavet, Guillaume Rabusseau
In class-incremental learning, a model learns continuously from a sequential data stream in which new classes occur. Existing methods often rely on static architectures that are ma…
Clustering-Oriented Representation Learning with Attractive-Repulsive Loss
Kian Kenyon-Dean, Andre Cianflone, Lucas Page-Caccia +3
The standard loss function used to train neural network classifiers, categorical cross-entropy (CCE), seeks to maximize accuracy on the training data; building useful representatio…
Hierarchical Methods of Moments
Matteo Ruffini, Guillaume Rabusseau, Borja Balle
Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to r…