24 citations · 69 across the 20 of their papers we have counts for
20 papers
RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
Jingru Lin, Chen Zhang, Stephen Y. Liu +1
Retrieval-Augmented Generation (RAG) mitigates key limitations of Large Language Models (LLMs)-such as factual errors, outdated knowledge, and hallucinations-by dynamically retriev…
Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models
Marcos Estecha-Garitagoitia, Chen Zhang, Mario Rodríguez-Cantelar +1
This paper provides preliminary results on exploring the task of performing turn-level data augmentation for dialogue system based on different types of commonsense relationships,…
Aligning Language Models Using Follow-up Likelihood as Reward Signal
Chen Zhang, Dading Chong, Feng Jiang +4
In natural human-to-human conversations, participants often receive feedback signals from one another based on their follow-up reactions. These reactions can include verbal respons…
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models
Chen Zhang, Chengguang Tang, Dading Chong +4
Mainstream approaches to aligning large language models (LLMs) heavily rely on human preference data, particularly when models require periodic updates. The standard process for it…
Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models
Yiming Chen, Chen Zhang, Danqing Luo +3
The automatic evaluation of natural language generation (NLG) systems presents a long-lasting challenge. Recent studies have highlighted various neural metrics that align well with…
CrossTune: Black-Box Few-Shot Classification with Label Enhancement
Danqing Luo, Chen Zhang, Yan Zhang +1
Training or finetuning large-scale language models (LLMs) requires substantial computation resources, motivating recent efforts to explore parameter-efficient adaptation to downstr…
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
Chen Zhang, Luis Fernando D'Haro, Yiming Chen +2
Automatic evaluation is an integral aspect of dialogue system research. The traditional reference-based NLG metrics are generally found to be unsuitable for dialogue assessment. Co…
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark
Chen Zhang, Luis Fernando D'Haro, Chengguang Tang +3
Recent advancements in reference-free learned metrics for open-domain dialogue evaluation have been driven by the progress in pre-trained language models and the availability of di…
Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4
Mario Rodríguez-Cantelar, Chen Zhang, Chengguang Tang +5
The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic…
Enhancing Black-Box Few-Shot Text Classification with Prompt-Based Data Augmentation
Danqing Luo, Chen Zhang, Jiahui Xu +4
Training or finetuning large-scale language models (LLMs) such as GPT-3 requires substantial computation resources, motivating recent efforts to explore parameter-efficient adaptat…
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Chatbots are expected to be knowledgeable across multiple domains, e.g. for daily chit-chat, exchange of information, and grounding in emotional situations. To effectively measure…
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment. However, they either perform turn-level evaluation…
A Focused Study on Sequence Length for Dialogue Summarization
Bin Wang, Chen Zhang, Chengwei Wei +1
Output length is critical to dialogue summarization systems. The dialogue summary length is determined by multiple factors, including dialogue complexity, summary objective, and pe…
Analyzing and Evaluating Faithfulness in Dialogue Summarization
Bin Wang, Chen Zhang, Yan Zhang +2
Dialogue summarization is abstractive in nature, making it suffer from factual errors. The factual correctness of summaries has the highest priority before practical applications.…
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
Shikib Mehri, Jinho Choi, Luis Fernando D'Haro +13
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggeste…
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation
Chen Zhang, Luis Fernando D'Haro, Thomas Friedrichs +1
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations. To measure…
Automatic Evaluation and Moderation of Open-domain Dialogue Systems
Chen Zhang, João Sedoc, Luis Fernando D'Haro +2
The development of Open-Domain Dialogue Systems (ODS)is a trending topic due to the large number of research challenges, large societal and business impact, and advances in the und…
Investigating the Impact of Pre-trained Language Models on Dialog Evaluation
Chen Zhang, Luis Fernando D'Haro, Yiming Chen +2
Recently, there is a surge of interest in applying pre-trained language models (Pr-LM) in automatic open-domain dialog evaluation. Pr-LMs offer a promising direction for addressing…
Revisiting Self-Training for Few-Shot Learning of Language Model
Yiming Chen, Yan Zhang, Chen Zhang +3
As unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. The question is how to effectively make use of such data. In…
DynaEval: Unifying Turn and Dialogue Level Evaluation
Chen Zhang, Yiming Chen, Luis Fernando D'Haro +4
A dialogue is essentially a multi-turn interaction among interlocutors. Effective evaluation metrics should reflect the dynamics of such interaction. Existing automatic metrics are…