41 citations · 123 across the 14 of their papers we have counts for
10 papers · 1 filter
Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Nishant Balepur, Paiheng Xu, Wei Ai +3
Multiple-choice question answering (MCQA) benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme, the combination of the response mod…
DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Data
Yuhang Zhou, Jing Zhu, Shengyi Qian +7
Large Language Models (LLMs) are increasingly aligned with human preferences through Reinforcement Learning from Human Feedback (RLHF). Among RLHF methods, Group Relative Policy Op…
Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of Topic Models
Zongxia Li, Lorena Calvo-Bartolomé, Alexander Hoyle +4
A common use of NLP is to facilitate the understanding of large document collections, with a shift from using traditional topic models to Large Language Models. Yet the effectivene…
Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation
Yuhang Zhou, Jing Zhu, Paiheng Xu +5
Large language models (LLMs) have significantly advanced various natural language processing tasks, but deploying them remains computationally expensive. Knowledge distillation (KD…
The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education
Paiheng Xu, Jing Liu, Nathan Jones +2
Assessing instruction quality is a fundamental component of any improvement efforts in the education system. However, traditional manual assessments are expensive, subjective, and…
Large Language Models and Causal Inference in Collaboration: A Survey
Xiaoyu Liu, Paiheng Xu, Junda Wu +10
Causal inference has shown potential in enhancing the predictive accuracy, fairness, robustness, and explainability of Natural Language Processing (NLP) models by capturing causal…
Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications
Yuhang Zhou, Paiheng Xu, Xiyao Wang +3
Emojis, which encapsulate semantics beyond mere words or phrases, have become prevalent in social network communications. This has spurred increasing scholarly interest in explorin…
Explore Spurious Correlations at the Concept Level in Language Models for Text Classification
Yuhang Zhou, Paiheng Xu, Xiaoyu Liu +3
Language models (LMs) have achieved notable success in numerous NLP tasks, employing both fine-tuning and in-context learning (ICL) methods. While language models demonstrate excep…
Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps
Fuxiao Liu, Paiheng Xu, Zongxia Li +2
We investigate the role of various demonstration components in the in-context learning (ICL) performance of large language models (LLMs). Specifically, we explore the impacts of gr…
Using Noisy Self-Reports to Predict Twitter User Demographics
Zach Wood-Doughty, Paiheng Xu, Xiao Liu +1
Computational social science studies often contextualize content analysis within standard demographics. Since demographics are unavailable on many social media platforms (e.g. Twit…