312 citations · 474 across the 7 of their papers we have counts for
7 papers
The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?
Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael +3
There are pronounced differences in the extent to which industrial and academic AI labs use computing resources. We provide a data-driven survey of the role of the compute divide i…
International Governance of Civilian AI: A Jurisdictional Certification Approach
Robert Trager, Ben Harack, Anka Reuel +9
This report describes trade-offs in the design of international governance arrangements for civilian artificial intelligence (AI) and presents one approach in detail. This approach…
Towards best practices in AGI safety and governance: A survey of expert opinion
Jonas Schuett, Noemi Dreksler, Markus Anderljung +4
A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve…
Will we run out of data? Limits of LLM scaling based on human-generated data
Pablo Villalobos, Anson Ho, Jaime Sevilla +3
We investigate the potential constraints on LLM scaling posed by the availability of public human-generated text data. We forecast the growing demand for training data based on cur…
Machine Learning Model Sizes and the Parameter Gap
Pablo Villalobos, Jaime Sevilla, Tamay Besiroglu +3
We study trends in model size of notable machine learning systems over time using a curated dataset. From 1950 to 2018, model size in language models increased steadily by seven or…
Compute Trends Across Three Eras of Machine Learning
Jaime Sevilla, Lennart Heim, Anson Ho +3
Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern Machine Learning (ML). In this paper we study trends in the most readily…
Measuring what Really Matters: Optimizing Neural Networks for TinyML
Lennart Heim, Andreas Biri, Zhongnan Qu +1
With the surge of inexpensive computational and memory resources, neural networks (NNs) have experienced an unprecedented growth in architectural and computational complexity. Intr…