Publications (67)
Learning Dynamic Feature Selection for Fast Sequential Prediction
Emma Strubell, Luke Vilnis, Kate Silverstein +1
Collage: Decomposable Rapid Prototyping for Information Extraction on Scientific PDFs
Sireesh Gururaja, Yueheng Zhang, Guannan Tang +6
The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures
Sheshera Mysore, Zach Jensen, Edward Kim +6
Just CHOP: Embarrassingly Simple LLM Compression
Ananya Harsh Jha, Tom Sherborne, Evan Pete Walsh +3
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters
Li Lucy, Suchin Gururangan, Luca Soldaini +4
Dependency Parsing with Dilated Iterated Graph CNNs
Emma Strubell, Andrew McCallum
Understanding the Effect of Model Compression on Social Bias in Large Language Models
Gustavo Gonçalves, Emma Strubell
Linguistically-Informed Self-Attention for Semantic Role Labeling
Emma Strubell, Patrick Verga, Daniel Andor +2
Misinformation by Omission: The Need for More Environmental Transparency in AI
Sasha Luccioni, Boris Gamazaychikov, Theo Alves da Costa +1
WiFiMod: Transformer-based Indoor Human Mobility Modeling using Passive Sensing
Amee Trivedi, Kate Silverstein, Emma Strubell +2
Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment
Jared Fernandez, Clara Na, Yonatan Bisk +2
BlueFin: Benchmarking LLM Agents on Financial Spreadsheets
Srivatsa Kundurthy, Clara Na, Colton Moraine +6
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment
Jared Fernandez, Jacob Kahn, Clara Na +2
Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolution
Nupoor Gandhi, Anjalie Field, Emma Strubell
Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints
Rajshekhar Das, Jonathan Francis, Sanket Vaibhav Mehta +3
International AI Safety Report
Yoshua Bengio, Sören Mindermann, Daniel Privitera +93
Energy and Policy Considerations for Deep Learning in NLP
Emma Strubell, Ananya Ganesh, Andrew McCallum
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training
Jared Fernandez, Luca Wehrstedt, Leonid Shamis +5
Carbon Connect: An Ecosystem for Sustainable Computing
Benjamin C. Lee, David Brooks, Arthur van Benthem +10
Making Scalable Meta Learning Practical
Sang Keun Choe, Sanket Vaibhav Mehta, Hwijeen Ahn +4
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
Clara Na, Sanket Vaibhav Mehta, Emma Strubell
Structured Extraction of Process Structure Properties Relationships in Materials Science
Amit K Verma, Zhisong Zhang, Junwon Seo +4
Source-Aware Training Enables Knowledge Attribution in Language Models
Muhammad Khalifa, David Wadden, Emma Strubell +4
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Luca Soldaini, Rodney Kinney, Akshita Bhagia +33
Stereotype or Personalization? User Identity Biases Chatbot Recommendations
Anjali Kantharuban, Jeremiah Milbauer, Maarten Sap +2
The Energy Cost of Execution-Idle in GPU Clusters
Yiran Lei, Jared Fernandez, Vasilis Kypriotis +4
Task Decomposition for Efficient Annotation
Nupoor Gandhi, Emma Strubell
OLMo: Accelerating the Science of Language Models
Dirk Groeneveld, Iz Beltagy, Pete Walsh +40
Multilingual Relation Extraction using Compositional Universal Schema
Patrick Verga, David Belanger, Emma Strubell +2
Bridging Fairness and Environmental Sustainability in Natural Language Processing
Marius Hessenthaler, Emma Strubell, Dirk Hovy +1
Kinetics: Rethinking Test-Time Scaling Laws
Ranajoy Sadhukhan, Zhuoming Chen, Haizhong Zheng +3
On-device Streaming Discrete Speech Units
Kwanghee Choi, Masao Someki, Emma Strubell +1
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation
Hao Peng, Qingqing Cao, Jesse Dodge +11
Energy Considerations of Large Language Model Inference and Efficiency Optimizations
Jared Fernandez, Clara Na, Vashisth Tiwari +3
Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models
Harnoor Dhingra, Preetiha Jayashanker, Sayali Moghe +1
Efficient Methods for Natural Language Processing: A Survey
Marcos Treviso, Ji-Ung Lee, Tianchu Ji +19
A Survey of Active Learning for Natural Language Processing
Zhisong Zhang, Emma Strubell, Eduard Hovy
SQuAT: Sharpness- and Quantization-Aware Training for BERT
Zheng Wang, Juncheng B Li, Shuhui Qu +2
Expert Routing with Synthetic Data for Continual Learning
Yewon Byun, Sanket Vaibhav Mehta, Saurabh Garg +4
To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering
Dheeru Dua, Emma Strubell, Sameer Singh +1
An Empirical Investigation of the Role of Pre-training in Lifelong Learning
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar +1
FicSim: A Dataset for Multi-Faceted Semantic Similarity in Long-Form Fiction
Natasha Johnson, Amanda Bertsch, Maria-Emil Deal +1
Holistically Evaluating the Environmental Impact of Creating Language Models
Jacob Morrison, Clara Na, Jared Fernandez +3
Power Hungry Processing: Watts Driving the Cost of AI Deployment?
Alexandra Sasha Luccioni, Yacine Jernite, Emma Strubell
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Edward Kim, Zach Jensen, Alexander van Grootel +8
Automatically Extracting Action Graphs from Materials Science Synthesis Procedures
Sheshera Mysore, Edward Kim, Emma Strubell +6
Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?
Emma Strubell, Andrew McCallum
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
Emma Strubell, Patrick Verga, David Belanger +1
Training for Fast Sequential Prediction Using Dynamic Feature Selection
Emma Strubell, Luke Vilnis, Andrew McCallum
From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate
Alexandra Sasha Luccioni, Emma Strubell, Kate Crawford
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
Sang Keun Choe, Hwijeen Ahn, Juhan Bae +11
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
Clara Na, Ian Magnusson, Ananya Harsh Jha +4
Error-aware Quantization through Noise Tempering
Zheng Wang, Juncheng B Li, Shuhui Qu +2
SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
Srivatsa Kundurthy, Clara Na, Michael Handley +5
Beyond Text: Characterizing Domain Expert Needs in Document Research
Sireesh Gururaja, Nupoor Gandhi, Jeremiah Milbauer +1
Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training
Zhisong Zhang, Emma Strubell, Eduard Hovy
Measuring the Carbon Intensity of AI in Cloud Instances
Jesse Dodge, Taylor Prewitt, Remi Tachet Des Combes +7
Empirically-Calibrated H100 Node Power Models for Reducing Uncertainty in AI Training Energy Estimation
Alex C. Newkirk, Jared Fernandez, Jonathan Koomey +4
To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing
Sireesh Gururaja, Amanda Bertsch, Clara Na +2
Energy and Carbon Considerations of Fine-Tuning BERT
Xiaorong Wang, Clara Na, Emma Strubell +2
The Hidden Cost of Thinking: Energy Use and Environmental Impact of LMs Beyond Pretraining
Jacob Morrison, Noah A. Smith, Emma Strubell
DSI++: Updating Transformer Memory with New Documents
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay +6
Attending to All Mention Pairs for Full Abstract Biological Relation Extraction
Patrick Verga, Emma Strubell, Ofer Shai +1
Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction
Patrick Verga, Emma Strubell, Andrew McCallum
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay +3
Gradient Localization Improves Lifelong Pretraining of Language Models
Jared Fernandez, Yonatan Bisk, Emma Strubell
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Ji-Ung Lee, Haritz Puerto, Betty van Aken +8