4 papers
When Sample Selection Bias Precipitates Model Collapse
Xinbao Qiao, Xianglong Du, Wei Liu +4
The proliferation of recursive training on synthetic data can alleviate data scarcity but risks model collapse, where repeated training erodes distributional tails and homogenizes…
More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensembles
Ruibo Chen, Yihan Wu, Xuehao Cui +2
Watermarking has emerged as a crucial technique for detecting and attributing content generated by large language models. While recent advancements have utilized watermark ensemble…
As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Files
Haodong Li, Jingqi Zhang, Xiao Cheng +3
The remarkable language ability of Large Language Models (LLMs) stems from extensive training on vast datasets, often including copyrighted material, which raises serious concerns…
Leave No TRACE: Black-box Detection of Copyrighted Dataset Usage in Large Language Models via Watermarking
Jingqi Zhang, Ruibo Chen, Yingqing Yang +3
Large Language Models (LLMs) are increasingly fine-tuned on smaller, domain-specific datasets to improve downstream performance. These datasets often contain proprietary or copyrig…