4 papers
CrowdAgent: Multi-Agent Managed Multi-Source Annotation System
Maosheng Qin, Renyu Zhu, Mingxuan Xia +8
High-quality annotated data is a cornerstone of modern Natural Language Processing (NLP). While recent methods begin to leverage diverse annotation sources-including Large Language…
Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation
Mingxuan Xia, Haobo Wang, Yixuan Li +4
Recently, Large Language Models (LLMs) have demonstrated significant potential for data annotation, markedly reducing the labor costs associated with downstream applications. Howev…
The Effects of Data Augmentation on Confidence Estimation for LLMs
Rui Wang, Renyu Zhu, Minmin Lin +4
Confidence estimation is crucial for reflecting the reliability of large language models (LLMs), particularly in the widely used closed-source models. Utilizing data augmentation f…
Rank Aggregation in Crowdsourcing for Listwise Annotations
Wenshui Luo, Haoyu Liu, Yongliang Ding +7
Rank aggregation through crowdsourcing has recently gained significant attention, particularly in the context of listwise ranking annotations. However, existing methods primarily f…