7 papers
Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?
Sho Hoshino, Ukyo Honda, Peinan Zhang
While self-consistency is known to improve performance on symbolic reasoning, its effect on the recall of encyclopedic knowledge is unclear due to a lack of targeted evaluation gro…
AdTEC: A Unified Benchmark for Evaluating Text Quality in Search Engine Advertising
Peinan Zhang, Yusuke Sakai, Masato Mita +2
With the increase in the fluency of ad texts automatically created by natural language generation technology, there is high demand to verify the quality of these creatives in a rea…
Out-of-the-Box Conditional Text Embeddings from Large Language Models
Kosuke Yamada, Peinan Zhang
Conditional text embedding is a proposed representation that captures the shift in perspective on texts when conditioned on a specific aspect. Previous methods have relied on exten…
LCTG Bench: LLM Controlled Text Generation Benchmark
Kentaro Kurihara, Masato Mita, Peinan Zhang +3
The rise of large language models (LLMs) has led to more diverse and higher-quality machine-generated text. However, their high expressive power makes it difficult to control outpu…
Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding
Ukyo Honda, Tatsushi Oka, Peinan Zhang +1
Recent models for natural language understanding are inclined to exploit simple patterns in datasets, commonly known as shortcuts. These shortcuts hinge on spurious correlations be…
FaithCAMERA: Construction of a Faithful Dataset for Ad Text Generation
Akihiko Kato, Masato Mita, Soichiro Murakami +3
In ad text generation (ATG), desirable ad text is both faithful and informative. That is, it should be faithful to the input document, while at the same time containing important i…