Publications (10)
Gated Linear Networks
Joel Veness, Tor Lattimore, David Budden +8
This paper presents a new family of backpropagation-free neural architectures, Gated Linear Networks (GLNs). What distinguishes GLNs from contemporary neural networks is the distri…
Decentralised AI Training and Inference with BlockTrain
Peter Toth, Dan Oprisa
Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters. This creates a structural advantage for hyperscalers and large centralize…
High-yield production of 2D crystals by wet-jet milling
Antonio Esau Del Rio Castillo, Vittorio Pellegrini, Alberto Ansaldo +17
Efficient and scalable production of two-dimensional (2D) materials is required to overcome technological hurdles towards the creation of a 2D-materials-based industry. Here, we pr…
Online Learning with Gated Linear Networks
Joel Veness, Tor Lattimore, Avishkar Bhoopchand +3
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our meth…
Criticality & Deep Learning I: Generally Weighted Nets
Dan Oprisa, Peter Toth
Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological…
Equivariant Hamiltonian Flows
Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins +1
This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformat…
Criticality & Deep Learning II: Momentum Renormalisation Group
Dan Oprisa, Peter Toth
Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learn…
On the zero mass limit of tagged particle diffusion in the 1-d Rayleigh-gas
Peter Balint, Balint Toth, Peter Toth
We consider the M -> 0 limit for tagged particle diffusion in a 1-dimensional Rayleigh-gas, studied originaly by Sinai and Soloveichik (1986), respectively by Szasz and Toth (1986)…
Hamiltonian Generative Networks
Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle +3
The Hamiltonian formalism plays a central role in classical and quantum physics. Hamiltonians are the main tool for modelling the continuous time evolution of systems with conserve…
Metriplector: From Field Theory to Neural Architecture
Dan Oprisa, Peter Toth
We present Metriplector, a neural architecture primitive in which the input configures an abstract physical system -- fields, sources, and operators -- and the dynamics of that sys…