5 papers · 1 filter
Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining
Yongan Yu, Shantam Raj, Jingwei Ni +3
Natural Language Processing (NLP) in the climate domain requires models to process heterogeneous text sources, including scientific literature, policy disclosures, and synthetic re…
Legal Domain Adaptation of Modern BERT Models
Dominik Stammbach, Peter Henderson
We investigate domain adaptation of modern BERT models in the legal domain. We further pre-train ModernBERT on all US court opinions using the masked language modeling objective. A…
AI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments
Kylie Zhang, Nimra Nadeem, Lucia Zheng +2
In oral arguments, judges probe attorneys with questions about the factual record, legal claims, and the strength of their arguments. To prepare for this questioning, both law scho…
What Is The Political Content in LLMs' Pre- and Post-Training Data?
Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach +1
Large language models (LLMs) reflect politically-slanted opinions in their generated text. Even though it is widely assumed that model behavior stem from training data, there has b…
NLP for Social Good: A Survey and Outlook of Challenges, Opportunities, and Responsible Deployment
Antonia Karamolegkou, Angana Borah, Eunjung Cho +30
Natural language processing (NLP) now shapes many aspects of our world, yet its potential for positive social impact is underexplored. This paper surveys work in ``NLP for Social G…