310 citations · 582 across the 61 of their papers we have counts for
12 papers · 1 filter
Understanding the Effect of Model Compression on Social Bias in Large Language Models
Gustavo Gonçalves, Emma Strubell
Large Language Models (LLMs) trained with self-supervision on vast corpora of web text fit to the social biases of that text. Without intervention, these social biases persist in t…
Power Hungry Processing: Watts Driving the Cost of AI Deployment?
Alexandra Sasha Luccioni, Yacine Jernite, Emma Strubell
Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (…
Energy and Carbon Considerations of Fine-Tuning BERT
Xiaorong Wang, Clara Na, Emma Strubell +2
Despite the popularity of the `pre-train then fine-tune' paradigm in the NLP community, existing work quantifying energy costs and associated carbon emissions has largely focused o…
Making Scalable Meta Learning Practical
Sang Keun Choe, Sanket Vaibhav Mehta, Hwijeen Ahn +4
Despite its flexibility to learn diverse inductive biases in machine learning programs, meta learning (i.e., learning to learn) has long been recognized to suffer from poor scalabi…
To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing
Sireesh Gururaja, Amanda Bertsch, Clara Na +2
NLP is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. In this work, we seek to understand how to shape our future by…
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation
Hao Peng, Qingqing Cao, Jesse Dodge +11
Rising computational demands of modern natural language processing (NLP) systems have increased the barrier to entry for cutting-edge research while posing serious environmental co…
Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models
Harnoor Dhingra, Preetiha Jayashanker, Sayali Moghe +1
Large Language Models (LLMs) are trained primarily on minimally processed web text, which exhibits the same wide range of social biases held by the humans who created that content.…
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Ji-Ung Lee, Haritz Puerto, Betty van Aken +8
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters. Large model sizes makes computational cos…
Just CHOP: Embarrassingly Simple LLM Compression
Ananya Harsh Jha, Tom Sherborne, Evan Pete Walsh +3
Large language models (LLMs) enable unparalleled few- and zero-shot reasoning capabilities but at a high computational footprint. A growing assortment of methods for compression pr…
Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training
Zhisong Zhang, Emma Strubell, Eduard Hovy
In this work we propose a pragmatic method that reduces the annotation cost for structured label spaces using active learning. Our approach leverages partial annotation, which redu…
Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints
Rajshekhar Das, Jonathan Francis, Sanket Vaibhav Mehta +3
Self-training based on pseudo-labels has emerged as a dominant approach for addressing conditional distribution shifts in unsupervised domain adaptation (UDA) for semantic segmenta…
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment
Jared Fernandez, Jacob Kahn, Clara Na +2
Increased focus on the computational efficiency of NLP systems has motivated the design of efficient model architectures and improvements to underlying hardware accelerators. Howev…