5 papers
Ishigaki-IDS: An Open-Weight Verifier-Aware Model for Information Delivery Specification Drafting in Building Information Modeling
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto +9
Building Information Modeling (BIM) projects require information requirements to be described as machine-checkable Information Delivery Specification (IDS) files in order to verify…
Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto +9
Building Information Modeling (BIM) projects increasingly use Information Delivery Specification (IDS) to formalize information requirements in a machine-checkable XML format. Beca…
SWE-InfraBench: Evaluating Language Models on Cloud Infrastructure Code
Natalia Tarasova, Enrique Balp-Straffon, Aleksei Iancheruk +10
Building infrastructure-as-code (IaC) in cloud computing is a critical task, underpinning the reliability, scalability, and security of modern software systems. Despite the remarka…
GenAI-DrawIO-Creator: A Framework for Automated Diagram Generation
Jinze Yu, Dayuan Jiang
Diagrams are crucial for communicating complex information, yet creating and modifying them remains a labor-intensive task. We present GenAI-DrawIO-Creator, a novel framework that…
STED and Consistency Scoring: A Framework for Evaluating LLM Structured Output Reliability
Guanghui Wang, Jinze Yu, Xing Zhang +5
Large Language Models (LLMs) are increasingly deployed for structured data generation, yet output consistency remains critical for production applications. We introduce a comprehen…