44 citations · 52 across the 7 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021
LMSOC: An Approach for Socially Sensitive Pretraining
Vivek Kulkarni, Shubhanshu Mishra, Aria Haghighi
While large-scale pretrained language models have been shown to learn effective linguistic representations for many NLP tasks, there remain many real-world contextual aspects of la…
cs.CL2021
Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Prediction
Shubhanshu Mishra, Aria Haghighi
We evaluate a simple approach to improving zero-shot multilingual transfer of mBERT on social media corpus by adding a pretraining task called translation pair prediction (TPP), wh…