489 citations · 3.7k across the 37 of their papers we have counts for
38 papers
Towards Compute-Optimal Transfer Learning
Massimo Caccia, Alexandre Galashov, Arthur Douillard +6
The field of transfer learning is undergoing a significant shift with the introduction of large pretrained models which have demonstrated strong adaptability to a variety of downst…
Neural Algorithmic Reasoning with Causal Regularisation
Beatrice Bevilacqua, Kyriacos Nikiforou, Borja Ibarz +5
Recent work on neural algorithmic reasoning has investigated the reasoning capabilities of neural networks, effectively demonstrating they can learn to execute classical algorithms…
Unified Scaling Laws for Routed Language Models
Aidan Clark, Diego de las Casas, Aurelia Guy +23
The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count. Here we study the scaling behaviors of Routing Networks: archite…
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann +25
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. With a trillion to…
Prune Responsibly
Michela Paganini
Irrespective of the specific definition of fairness in a machine learning application, pruning the underlying model affects it. We investigate and document the emergence and exacer…
Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
Michela Paganini, Jessica Zosa Forde
The observation of sparse trainable sub-networks within over-parametrized networks - also known as Lottery Tickets (LTs) - has prompted inquiries around their trainability, scaling…
dagger: A Python Framework for Reproducible Machine Learning Experiment Orchestration
Michela Paganini, Jessica Zosa Forde
Many research directions in machine learning, particularly in deep learning, involve complex, multi-stage experiments, commonly involving state-mutating operations acting on models…
Streamlining Tensor and Network Pruning in PyTorch
Michela Paganini, Jessica Forde
In order to contrast the explosion in size of state-of-the-art machine learning models that can be attributed to the empirical advantages of over-parametrization, and due to the ne…
On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks
Michela Paganini, Jessica Forde
We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze diff…
Transverse momentum and process dependent azimuthal anisotropies in TeV +Pb collisions with the ATLAS detector
ATLAS Collaboration
The azimuthal anisotropy of charged particles produced in TeV +Pb collisions is measured with the ATLAS detector at the LHC. The data correspond to…
Evidence for electroweak production of two jets in association with a pair in collisions at TeV with the ATLAS detector
ATLAS Collaboration
Evidence for electroweak production of two jets in association with a pair in TeV protonproton collisions at the Large Hadron Collider is presented. The ana…
Combined measurements of Higgs boson production and decay using up to fb of proton-proton collision data at 13 TeV collected with the ATLAS experiment
ATLAS Collaboration
Combined measurements of Higgs boson production cross sections and branching fractions are presented. The combination is based on the analyses of the Higgs boson decay modes $H \to…
Search for electroweak production of charginos and sleptons decaying into final states with two leptons and missing transverse momentum in TeV collisions using the ATLAS detector
ATLAS Collaboration
A search for the electroweak production of charginos and sleptons decaying into final states with two electrons or muons is presented. The analysis is based on 139 fb of pro…
Search for heavy neutral Higgs bosons produced in association with -quarks and decaying into -quarks at TeV with the ATLAS detector
ATLAS Collaboration
A search for heavy neutral Higgs bosons produced in association with one or two -quarks and decaying to -quark pairs is presented using 27.8 fb of TeV pr…
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
Ari S. Morcos, Haonan Yu, Michela Paganini +1
The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately…
The Scientific Method in the Science of Machine Learning
Jessica Zosa Forde, Michela Paganini
In the quest to align deep learning with the sciences to address calls for rigor, safety, and interpretability in machine learning systems, this contribution identifies key missing…
Search for a right-handed gauge boson decaying into a high-momentum heavy neutrino and a charged lepton in collisions with the ATLAS detector at TeV
ATLAS Collaboration
A search for a right-handed gauge boson , decaying into a boosted right-handed heavy neutrino , in the framework of Left-Right Symmetric Models is p…
Fluctuations of anisotropic flow in Pb+Pb collisions at TeV with the ATLAS detector
ATLAS Collaboration
Multi-particle azimuthal cumulants are measured as a function of centrality and transverse momentum using 470 b of Pb+Pb collisions at TeV w…
Machine Learning Solutions for High Energy Physics: Applications to Electromagnetic Shower Generation, Flavor Tagging, and the Search for di-Higgs Production
Michela Paganini
This thesis demonstrate the efficacy of designing and developing machine learning (ML) algorithms to selected use cases that encompass many of the outstanding challenges in the fie…
Comparison of fragmentation functions for light-quark- and gluon-dominated jets from and Pb+Pb collisions in ATLAS
ATLAS Collaboration
Charged-particle fragmentation functions for jets azimuthally balanced by a high-transverse-momentum, prompt, isolated photon are measured in pb of and nb$^…
Electron reconstruction and identification in the ATLAS experiment using the 2015 and 2016 LHC proton-proton collision data at = 13 TeV
ATLAS Collaboration
Algorithms used for the reconstruction and identification of electrons in the central region of the ATLAS detector at the Large Hadron Collider (LHC) are presented in this paper; t…
Search for scalar resonances decaying into in events with and without -tagged jets produced in proton-proton collisions at TeV with the ATLAS detector
ATLAS Collaboration
A search for a narrow scalar resonance decaying into an opposite-sign muon pair produced in events with and without -tagged jets is presented in this paper. The search uses 36.1…
Measurement of the and cross sections in proton-proton collisions at TeV with the ATLAS detector
ATLAS Collaboration
A measurement of the associated production of a top-quark pair () with a vector boson (, ) in proton-proton collisions at a center-of-mass energy of 13 TeV is prese…