202 citations · 391 across the 9 of their papers we have counts for
9 papers
A Survey on Evaluation of Large Language Models
Yupeng Chang, Xu Wang, Jindong Wang +13
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to…
Measuring Consistency in Text-based Financial Forecasting Models
Linyi Yang, Yingpeng Ma, Yue Zhang
Financial forecasting has been an important and active area of machine learning research, as even the most modest advantage in predictive accuracy can be parlayed into significant…
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization
Yidong Wang, Zhuohao Yu, Zhengran Zeng +10
Instruction tuning large language models (LLMs) remains a challenging task, owing to the complexity of hyperparameter selection and the difficulty involved in evaluating the tuned…
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Kaijie Zhu, Jindong Wang, Jiaheng Zhou +8
The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this…
RFiD: Towards Rational Fusion-in-Decoder for Open-Domain Question Answering
Cunxiang Wang, Haofei Yu, Yue Zhang
Open-Domain Question Answering (ODQA) systems necessitate a reader model capable of generating answers by simultaneously referring to multiple passages. Although representative mod…
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future
Linyi Yang, Yaoxiao Song, Xuan Ren +6
Machine learning (ML) systems in natural language processing (NLP) face significant challenges in generalizing to out-of-distribution (OOD) data, where the test distribution differ…
Evaluating Open-QA Evaluation
Cunxiang Wang, Sirui Cheng, Qipeng Guo +7
This study focuses on the evaluation of the Open Question Answering (Open-QA) task, which can directly estimate the factuality of large language models (LLMs). Current automatic ev…
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Fengji Zhang, Bei Chen, Yue Zhang +6
The task of repository-level code completion is to continue writing the unfinished code based on a broader context of the repository. While for automated code completion tools, it…
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
Jindong Wang, Xixu Hu, Wenxin Hou +10
ChatGPT is a recent chatbot service released by OpenAI and is receiving increasing attention over the past few months. While evaluations of various aspects of ChatGPT have been don…