310 citations · 582 across the 61 of their papers we have counts for
68 papers
TractorBeam: Personalized AI Sensemaking Support via Collaborative Machine Annotation
Sireesh Gururaja, Jordan Taylor, Emma Strubell
Language model-based systems which allow asking questions of documents have become popular tools for sensemaking. Despite their implied capability, these systems still suffer from…
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1
With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associatio…
Task Decomposition for Efficient Annotation
Nupoor Gandhi, Emma Strubell
High-quality annotations of structured representations are expensive to collect over large corpora. Manual annotation of structure is laborious, and model-based annotation, althoug…
Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment
Jared Fernandez, Clara Na, Yonatan Bisk +2
Proper accounting of the energy requirements and environmental impact of artificial intelligence (AI) systems is necessary for researchers, developers, policy makers, and users to…
BlueFin: Benchmarking LLM Agents on Financial Spreadsheets
Srivatsa Kundurthy, Clara Na, Colton Moraine +6
We present BlueFin, a benchmark that tasks large language model (LLM) agents with synthesis, manipulation, and comprehension tasks over spreadsheet workbooks in the professional fi…
The Hidden Cost of Thinking: Energy Use and Environmental Impact of LMs Beyond Pretraining
Jacob Morrison, Noah A. Smith, Emma Strubell
Modern language model development extends far beyond pretraining, yet environmental reporting remains narrowly focused on the cost of training a single final model. In this work, w…
The Energy Cost of Execution-Idle in GPU Clusters
Yiran Lei, Jared Fernandez, Vasilis Kypriotis +4
GPUs are becoming a major contributor to data center power, yet unlike CPUs, they can remain at high power even when visible activity is near zero. We call this state execution-idl…
SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
Srivatsa Kundurthy, Clara Na, Michael Handley +5
We consider the task of end-to-end spreadsheet generation, where language models produce spreadsheet artifacts to satisfy users' explicit and implicit constraints, specified in nat…
FicSim: A Dataset for Multi-Faceted Semantic Similarity in Long-Form Fiction
Natasha Johnson, Amanda Bertsch, Maria-Emil Deal +1
As language models become capable of processing increasingly long and complex texts, there has been growing interest in their application within computational literary studies. How…
Kinetics: Rethinking Test-Time Scaling Laws
Ranajoy Sadhukhan, Zhuoming Chen, Haizhong Zheng +3
We rethink test-time scaling laws from a practical efficiency perspective, revealing that the effectiveness of smaller models is significantly overestimated. Prior work, grounded i…
Misinformation by Omission: The Need for More Environmental Transparency in AI
Sasha Luccioni, Boris Gamazaychikov, Theo Alves da Costa +1
In recent years, Artificial Intelligence (AI) models have grown in size and complexity, driving greater demand for computational power and natural resources. In parallel to this tr…
Empirically-Calibrated H100 Node Power Models for Reducing Uncertainty in AI Training Energy Estimation
Alex C. Newkirk, Jared Fernandez, Jonathan Koomey +4
As AI's energy demand continues to grow, it is critical to enhance the understanding of characteristics of this demand, to improve grid infrastructure planning and environmental as…
On-device Streaming Discrete Speech Units
Kwanghee Choi, Masao Someki, Emma Strubell +1
Discrete speech units (DSUs) are derived from clustering the features of self-supervised speech models (S3Ms). DSUs offer significant advantages for on-device streaming speech appl…
Energy Considerations of Large Language Model Inference and Efficiency Optimizations
Jared Fernandez, Clara Na, Vashisth Tiwari +3
As large language models (LLMs) scale in size and adoption, their computational and environmental costs continue to rise. Prior benchmarking efforts have primarily focused on laten…
Beyond Text: Characterizing Domain Expert Needs in Document Research
Sireesh Gururaja, Nupoor Gandhi, Jeremiah Milbauer +1
Working with documents is a key part of almost any knowledge work, from contextualizing research in a literature review to reviewing legal precedent. Recently, as their capabilitie…
Structured Extraction of Process Structure Properties Relationships in Materials Science
Amit K Verma, Zhisong Zhang, Junwon Seo +4
With the advent of large language models (LLMs), the vast unstructured text within millions of academic papers is increasingly accessible for materials discovery, although signific…
Holistically Evaluating the Environmental Impact of Creating Language Models
Jacob Morrison, Clara Na, Jared Fernandez +3
As the performance of artificial intelligence systems has dramatically increased, so too has the environmental impact of creating these systems. While many model developers release…
International AI Safety Report
Yoshua Bengio, Sören Mindermann, Daniel Privitera +93
The first International AI Safety Report comprehensively synthesizes the current evidence on the capabilities, risks, and safety of advanced AI systems. The report was mandated by…
From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate
Alexandra Sasha Luccioni, Emma Strubell, Kate Crawford
As the climate crisis deepens, artificial intelligence (AI) has emerged as a contested force: some champion its potential to advance renewable energy, materials discovery, and larg…
Expert Routing with Synthetic Data for Continual Learning
Yewon Byun, Sanket Vaibhav Mehta, Saurabh Garg +4
In many real-world settings, regulations and economic incentives permit the sharing of models but not data across institutional boundaries. In such scenarios, practitioners might h…
Gradient Localization Improves Lifelong Pretraining of Language Models
Jared Fernandez, Yonatan Bisk, Emma Strubell
Large Language Models (LLMs) trained on web-scale text corpora have been shown to capture world knowledge in their parameters. However, the mechanism by which language models store…
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training
Jared Fernandez, Luca Wehrstedt, Leonid Shamis +5
Dramatic increases in the capabilities of neural network models in recent years are driven by scaling model size, training data, and corresponding computational resources. To devel…
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
Clara Na, Ian Magnusson, Ananya Harsh Jha +4
Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance. However, a thorough data ablation study exploring large sets of c…
Collage: Decomposable Rapid Prototyping for Information Extraction on Scientific PDFs
Sireesh Gururaja, Yueheng Zhang, Guannan Tang +6
Recent years in NLP have seen the continued development of domain-specific information extraction tools for scientific documents, alongside the release of increasingly multimodal p…