9 papers
BayesPO: Bayesian Prompt Optimization via Parallel-Tempered Gradient-Guided Discrete MCMC
Junjie Zhou, Zhijian Ou
Prompt optimization adapts large language models (LLMs) without updating model parameters, but many automatic prompt optimizers remain heuristic search procedures over candidate in…
GEIS: A Generation-Evaluation-Improvement Loop of Agent Skills for Long-Form Article Generation
Jiale Zhang, Juntao Hu, Zhijian Ou
Long-form article generation remains difficult for large language models because it combines long context, long instructions, and long outputs. Existing multi-agent pipelines such…
Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximation
Hongyu Cao, Yuxuan Wu, Yucheng Cai +2
Retrieval-augmented generation (RAG) has become a widely recognized paradigm to combine parametric memory with non-parametric memories. An RAG model consists of two serial connecti…
Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems
Yucheng Cai, Yuxuan Wu, Yi Huang +2
Large language models (LLMs) have recently been applied to dialog systems. Despite making progress, LLMs are prone to errors in knowledge-intensive scenarios. Recently, approaches…
Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems
Yucheng Cai, Ke Li, Yi Huang +2
A retriever, which retrieves relevant knowledge pieces from a knowledge base given a context, is an important component in many natural language processing (NLP) tasks. Retrievers…
An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought
Yuetong Zhao, Hongyu Cao, Xianyu Zhao +1
Since the launch of ChatGPT at the end of 2022, generative dialogue models represented by ChatGPT have quickly become essential tools in daily life. As user expectations increase,…
The 2nd FutureDial Challenge: Dialog Systems with Retrieval Augmented Generation (FutureDial-RAG)
Yucheng Cai, Si Chen, Yuxuan Wu +3
Recently, increasing research interests have focused on retrieval augmented generation (RAG) to mitigate hallucination for large language models (LLMs). Following this trend, we la…
Prompt Pool based Class-Incremental Continual Learning for Dialog State Tracking
Hong Liu, Yucheng Cai, Yuan Zhou +3
Continual learning is crucial for dialog state tracking (DST) in dialog systems, since requirements from users for new functionalities are often encountered. However, most of exist…
UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt
Yucheng Cai, Wentao Ma, Yuchuan Wu +4
Recent research has shown that multi-task pre-training greatly improves the model's robustness and transfer ability, which is crucial for building a high-quality dialog system. How…