5 papers
To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation
John Chen, Sihan Cheng, Can Gurkan +1
Large language models (LLMs) are increasingly deployed as long-horizon agents with decision-making capacities. While LLMs can show ethical competence on dilemmas such as trolley pr…
A Computational Method for Measuring "Open Codes" in Qualitative Analysis
John Chen, Alexandros Lotsos, Sihan Cheng +7
Qualitative analysis is critical to understanding human datasets in many social science disciplines. A central method in this process is inductive coding, where researchers identif…
CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V
John Chen, Sihan Cheng, Can Gurkan +1
Evaluating strategic decision-making in LLM-based agents requires generative, competitive, and longitudinal environments, yet few benchmarks provide all three, and fewer still offe…
Vox Deorum: A Hybrid LLM Architecture for 4X / Grand Strategy Game AI -- Lessons from Civilization V
John Chen, Sihan Cheng, Can Gurkan +2
Large Language Models' capacity to reason in natural language makes them uniquely promising for 4X and grand strategy games, enabling more natural human-AI gameplay interactions su…
Processes Matter: How ML/GAI Approaches Could Support Open Qualitative Coding of Online Discourse Datasets
John Chen, Alexandros Lotsos, Grace Wang +4
Open coding, a key inductive step in qualitative research, discovers and constructs concepts from human datasets. However, capturing extensive and nuanced aspects or "coding moment…