125 citations · 238 across the 12 of their papers we have counts for
13 papers
Perch 2.0 transfers 'whale' to underwater tasks
Andrea Burns, Lauren Harrell, Bart van Merriënboer +3
Perch 2.0 is a supervised bioacoustics foundation model pretrained on 14,597 species, including birds, mammals, amphibians, and insects, and has state-of-the-art performance on mul…
The Search for Squawk: Agile Modeling in Bioacoustics
Vincent Dumoulin, Otilia Stretcu, Jenny Hamer +16
Passive acoustic monitoring (PAM) has shown great promise in helping ecologists understand the health of animal populations and ecosystems. However, extracting insights from millio…
BIRB: A Generalization Benchmark for Information Retrieval in Bioacoustics
Jenny Hamer, Eleni Triantafillou, Bart van Merriënboer +4
The ability for a machine learning model to cope with differences in training and deployment conditions--e.g. in the presence of distribution shift or the generalization to new cla…
GradMax: Growing Neural Networks using Gradient Information
Utku Evci, Bart van Merriënboer, Thomas Unterthiner +2
The architecture and the parameters of neural networks are often optimized independently, which requires costly retraining of the parameters whenever the architecture is modified.…
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis
Courtney Paquette, Bart van Merriënboer, Elliot Paquette +1
Average-case analysis computes the complexity of an algorithm averaged over all possible inputs. Compared to worst-case analysis, it is more representative of the typical behavior…
Fast Training of Sparse Graph Neural Networks on Dense Hardware
Matej Balog, Bart van Merriënboer, Subhodeep Moitra +2
Graph neural networks have become increasingly popular in recent years due to their ability to naturally encode relational input data and their ability to scale to large graphs by…
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas, Fabian Pedregosa, Bart van Merriënboer +3
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most pre…
Automatic differentiation in ML: Where we are and where we should be going
Bart van Merriënboer, Olivier Breuleux, Arnaud Bergeron +1
We review the current state of automatic differentiation (AD) for array programming in machine learning (ML), including the different approaches such as operator overloading (OO) a…
Tangent: Automatic differentiation using source-code transformation for dynamically typed array programming
Bart van Merriënboer, Dan Moldovan, Alexander B Wiltschko
The need to efficiently calculate first- and higher-order derivatives of increasingly complex models expressed in Python has stressed or exceeded the capabilities of available tool…
Tangent: Automatic Differentiation Using Source Code Transformation in Python
Bart van Merriënboer, Alexander B. Wiltschko, Dan Moldovan
Automatic differentiation (AD) is an essential primitive for machine learning programming systems. Tangent is a new library that performs AD using source code transformation (SCT)…
Multiscale sequence modeling with a learned dictionary
Bart van Merriënboer, Amartya Sanyal, Hugo Larochelle +1
We propose a generalization of neural network sequence models. Instead of predicting one symbol at a time, our multi-scale model makes predictions over multiple, potentially overla…
Theano: A Python framework for fast computation of mathematical expressions
The Theano Development Team, Rami Al-Rfou, Guillaume Alain +110
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has bee…
Blocks and Fuel: Frameworks for deep learning
Bart van Merriënboer, Dzmitry Bahdanau, Vincent Dumoulin +4
We introduce two Python frameworks to train neural networks on large datasets: Blocks and Fuel. Blocks is based on Theano, a linear algebra compiler with CUDA-support. It facilitat…