6 papers
Averaging Bias: Human Faithfulness Annotations are not Locally Faithful
Huajian Zhang, Yiyang Feng, Jiawei Zhou
Evaluation of faithfulness of text summarization treats a model generated summary as faithful only if every of its sentences is supported by the source document: a strict conjuncti…
Evaluating Reasoning Fidelity in Visual Text Generation
Jiajun Hong, Jiawei Zhou
Recent text-to-image (T2I) models can render highly legible and well-structured text within images, enabling applications including document generation and slide generation. Howeve…
Lost in Instructions: Study of Blind Users' Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products
Monalika Padma Reddy, Aruna Balasubramanian, Jiawei Zhou +3
AI tools like ChatGPT and Be-My-AI are increasingly being used by blind individuals. Although prior work has explored their use in some Do-It-Yourself (DIY) tasks by blind individu…
Finding the Signal in the Noise: An Exploratory Study on Assessing the Effectiveness of AI and Accessibility Forums for Blind Users' Support Needs
Satwik Ram Kodandaram, Jiawei Zhou, Xiaojun Bi +2
Accessibility forums and, more recently, generative AI tools have become vital resources for blind users seeking solutions to computer-interaction issues and learning about new ass…
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge
Yiyang Feng, Zeming Chen, Haotian Wu +2
A common solution for mitigating outdated or incorrect information in Large Language Models (LLMs) is to provide updated facts in-context or through knowledge editing. However, the…
Unraveling Misinformation Propagation in LLM Reasoning
Yiyang Feng, Yichen Wang, Shaobo Cui +3
Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, positioning them as promising tools for supporting human problem-solving. However, what happens…