6 papers · 1 filter
ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding
Mingyang Rao, Kehua Feng, Zhihui Zhu +4
While Large Language Models (LLMs) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams. We identi…
SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
Keyang Xuan, Pengda Wang, Chongrui Ye +3
Large language models (LLMs) are increasingly evaluated in interactive environments to test their social intelligence. However, existing benchmarks often assume idealized communica…
The Ramon Llull's Thinking Machine for Automated Ideation
Xinran Zhao, Boyuan Zheng, Chenglei Si +8
This paper revisits Ramon Llull's Ars combinatoria - a medieval framework for generating knowledge through symbolic recombination - as a conceptual foundation for building a modern…
ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities
Zhaochen Hong, Haofei Yu, Jiaxuan You
Evaluating consistency in large language models (LLMs) is crucial for ensuring reliability, particularly in complex, multi-step interactions between humans and LLMs. Traditional se…
SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agents
Kunlun Zhu, Jiaxun Zhang, Ziheng Qi +6
Recent advancements in large language model (LLM) agents have significantly accelerated scientific discovery automation, yet concurrently raised critical ethical and safety concern…
Table as Thought: Exploring Structured Thoughts in LLM Reasoning
Zhenjie Sun, Naihao Deng, Haofei Yu +1
Large language models' reasoning abilities benefit from methods that organize their thought processes, such as chain-of-thought prompting, which employs a sequential structure to g…