most citedA Survey on Evaluation of Large Language Models

202 citations · 658 across the 25 of their papers we have counts for

collaborators

25 papers

cs.LG2023★ 5 cited

Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks

Hao Chen, Jindong Wang, Ankit Shah +5

Pre-training on large-scale datasets and then fine-tuning on downstream tasks have become a standard practice in deep learning. However, pre-training data often contain label noise…

cs.LG2023★ 2 cited

DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization

Wang Lu, Jindong Wang, Xinwei Sun +4

Time series remains one of the most challenging modalities in machine learning research. The out-of-distribution (OOD) detection and generalization on time series tend to suffer du…

cs.LG2023★ 2 cited

Improving Generalization of Adversarial Training via Robust Critical Fine-Tuning

Kaijie Zhu, Jindong Wang, Xixu Hu +2

Deep neural networks are susceptible to adversarial examples, posing a significant security risk in critical applications. Adversarial Training (AT) is a well-established technique…

cs.LG2023

Frustratingly Easy Model Generalization by Dummy Risk Minimization

Juncheng Wang, Jindong Wang, Xixu Hu +2

Empirical risk minimization (ERM) is a fundamental machine learning paradigm. However, its generalization ability is limited in various tasks. In this paper, we devise Dummy Risk M…

cs.CL2023★ 61 cited

Large Language Models Understand and Can be Enhanced by Emotional Stimuli

Cheng Li, Jindong Wang, Yixuan Zhang +6

Emotional intelligence significantly impacts our daily behaviors and interactions. Although Large Language Models (LLMs) are increasingly viewed as a stride toward artificial gener…

cs.CL2023★ 202 cited

A Survey on Evaluation of Large Language Models

Yupeng Chang, Xu Wang, Jindong Wang +13

Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to…

cs.CV2023

Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning

Xin Qin, Jindong Wang, Shuo Ma +4

Human activity recognition (HAR) is a time series classification task that focuses on identifying the motion patterns from human sensor readings. Adequate data is essential but a m…

cs.CL2023★ 29 cited

PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization

Yidong Wang, Zhuohao Yu, Zhengran Zeng +10

Instruction tuning large language models (LLMs) remains a challenging task, owing to the complexity of hyperparameter selection and the difficulty involved in evaluating the tuned…

cs.CL2023★ 51 cited

PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

Kaijie Zhu, Jindong Wang, Jiaheng Zhou +8

The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this…

cs.LG2023★ 8 cited

Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations

Hao Chen, Ankit Shah, Jindong Wang +6

Learning with reduced labeling standards, such as noisy label, partial label, and multiple label candidates, which we generically refer to as \textit{imprecise} labels, is a common…

cs.AI2023★ 1 cited

Exploring Vision-Language Models for Imbalanced Learning

Yidong Wang, Zhuohao Yu, Jindong Wang +6

Vision-Language models (VLMs) that use contrastive language-image pre-training have shown promising zero-shot classification performance. However, their performance on imbalanced d…

cs.CV2023★ 3 cited

IRGen: Generative Modeling for Image Retrieval

Yidan Zhang, Ting Zhang, Dong Chen +11

While generative modeling has become prevalent across numerous research fields, its integration into the realm of image retrieval remains largely unexplored and underjustified. In…

cs.LG2023★ 19 cited

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

Wang Lu, Xixu Hu, Jindong Wang +1

Federated learning (FL) has emerged as a new paradigm for privacy-preserving computation in recent years. Unfortunately, FL faces two critical challenges that hinder its actual per…

cs.AI2023★ 92 cited

On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective

Jindong Wang, Xixu Hu, Wenxin Hou +10

ChatGPT is a recent chatbot service released by OpenAI and is receiving increasing attention over the past few months. While evaluations of various aspects of ChatGPT have been don…

cs.LG2023★ 90 cited

SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning

Hao Chen, Ran Tao, Yue Fan +6

The critical challenge of Semi-Supervised Learning (SSL) is how to effectively leverage the limited labeled data and massive unlabeled data to improve the model's generalization pe…

cs.CL2022★ 1 cited

Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation

Jinyi Hu, Xiaoyuan Yi, Wenhao Li +2

Variational Auto-Encoder (VAE) has been widely adopted in text generation. Among many variants, recurrent VAE learns token-wise latent variables with each conditioned on the preced…

cs.CV2022★ 4 cited

An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learning

Hao Chen, Yue Fan, Yidong Wang +5

Semi-supervised learning (SSL) has shown great promise in leveraging unlabeled data to improve model performance. While standard SSL assumes uniform data distribution, we consider…

cs.CL2022★ 10 cited

GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective

Linyi Yang, Shuibai Zhang, Libo Qin +6

Pre-trained language models (PLMs) are known to improve the generalization performance of natural language understanding models by leveraging large amounts of data during the pre-t…

cs.CV2022★ 3 cited

FIXED: Frustratingly Easy Domain Generalization with Mixup

Wang Lu, Jindong Wang, Han Yu +4

Domain generalization (DG) aims to learn a generalizable model from multiple training domains such that it can perform well on unseen target domains. A popular strategy is to augme…

cs.LG2022★ 12 cited

Out-of-Distribution Representation Learning for Time Series Classification

Wang Lu, Jindong Wang, Xinwei Sun +2

Time series classification is an important problem in real world. Due to its non-stationary property that the distribution changes over time, it remains challenging to build models…

cs.LG2022★ 1 cited

Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution

Wang Lu, Jindong Wang, Yidong Wang +1

The distribution shifts between training and test data typically undermine the performance of models. In recent years, lots of work pays attention to domain generalization (DG) whe…

cs.LG2022

Equivariant Disentangled Transformation for Domain Generalization under Combination Shift

Yivan Zhang, Jindong Wang, Xing Xie +1

Machine learning systems may encounter unexpected problems when the data distribution changes in the deployment environment. A major reason is that certain combinations of domains…

cs.LG2022

Domain-Specific Risk Minimization for Out-of-Distribution Generalization

Yi-Fan Zhang, Jindong Wang, Jian Liang +5

Recent domain generalization (DG) approaches typically use the hypothesis learned on source domains for inference on the unseen target domain. However, such a hypothesis can be arb…

cs.LG2022★ 42 cited

USB: A Unified Semi-supervised Learning Benchmark for Classification

Yidong Wang, Hao Chen, Yue Fan +19

Semi-supervised learning (SSL) improves model generalization by leveraging massive unlabeled data to augment limited labeled samples. However, currently, popular SSL evaluation pro…