19 papers
Synthetic Scenario Generation for Evaluation of Industry 4.0 Agents
Sagar Chethan Kumar, Rohith Kanathur, Dhaval Patel +1
Industrial agent benchmarks require realistic evaluation scenarios that integrate telemetry, failure modes, maintenance records, and domain standards. However, existing benchmarks…
Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents
Dhaval C. Patel, Kaoutar El Maghraoui, Shuxin Lin +58
Agent benchmarks are growing fast, but no single benchmark touches more than four or five of the dimensions that deployment exposes. This paper aggregates the largest coordinated d…
DynAMO:Dynamic Asset Management Orchestration via Topological Multi-Agent Scheduling
Kanishk Kushwaha, Vikrant Vinod Bansode, Harsh Vardhan +1
While LLM-powered agents offer end-to-end automation for industrial asset lifecycles, real-world Industry 4.0 deployment is hindered by latency, concurrency instability, and safety…
EvoFlux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents
Kushal Raj Bhandari, Ling Yue, Ching-Yun Ko +4
Small language models (1.5B-4B parameters) offer low-cost, low-latency tool use but often produce plausible plans that fail during execution due to unavailable tools, missing argum…
Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows
Harshada Badave, Santosh Borse, Andrea Gomez +6
Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination benchmarks still evaluate on…
Internalizing Tool Knowledge in Small Language Models via QLoRA Fine-Tuning
Yuval Shemla, Ayal Yakobe, Tanmay Agarwal +2
Large language models are increasingly used as planning components in agentic systems, but current tool-use pipelines often require full tool schemas to be included in every prompt…
Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines
Alimurtaza Mustafa Merchant, Krish Veera, Sajal Kumar Goyla +3
Industrial asset operations workflows are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure modes, forecasting tools, an…
SPIN: Structural LLM Planning via Iterative Navigation for Industrial Tasks
Yusuke Ozaki, Dhaval Patel
Industrial LLM agent systems often separate planning from execution, yet LLM planners frequently produce structurally invalid or unnecessarily long workflows, leading to brittle fa…
Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds
Prateek Biswas, Dhaval Patel, Vedant Khandelwal +2
Multiple-choice QA benchmarks usually evaluate small language models (SLMs) as direct answerers, but deployed language-model systems increasingly rely on external scaffolds such as…
MCP-Cosmos: World Model-Augmented Agents for Complex Task Execution in MCP Environments
Giridhar Ganapavarapu, Dhaval Patel
The Model Context Protocol (MCP) has unified the interface between Large Language Models (LLMs) and external tools, yet a fundamental gap remains in how agents conceptualize the en…
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules
Devin Yasith De Silva, Dhaval Patel, Christodoulos Constantinides +7
Monitoring complex industrial assets relies on engineer-authored symbolic rules that trigger based on sensor conditions and prompt technicians to perform corrective actions. The bo…
Results and Retrospective Analysis of the CODS 2025 AssetOpsBench Challenge
Dhaval Patel, Chathurangi Shyalika, Suryanarayana Reddy Yarrabothula +4
Competition retrospectives are useful when they explain what a leaderboard measured, how hidden evaluation changed conclusions, and which design patterns were rewarded. We revisit…
IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenance
Chathurangi Shyalika, Dhaval Patel, Amit Sheth
Industrial maintenance environments increasingly rely on AI systems to assist operators in understanding asset behavior, diagnosing failures, and evaluating interventions. Although…
PHMForge: Evaluating LLM Agents on Industrial Prognostics through MCP-Native, Algorithm-Grounded Tools
Yusheng Li, Tianjun Feng, Yunfeng Chen +6
LLM agents are beginning to invoke industrial asset-management tools through the Model Context Protocol (MCP), yet whether they can act reliably on this substrate for safety-critic…
Fine-Tuned Thoughts: Leveraging Chain-of-Thought Reasoning for Industrial Asset Health Monitoring
Shuxin Lin, Dhaval Patel, Christodoulos Constantinides
Small Language Models (SLMs) are becoming increasingly popular in specialized fields, such as industrial applications, due to their efficiency, lower computational requirements, an…
Towards Building General Purpose Embedding Models for Industry 4.0 Agents
Christodoulos Constantinides, Shuxin Lin, Dhaval Patel
In this work we focus on improving language models' understanding for asset maintenance to guide the engineer's decisions and minimize asset downtime. Given a set of tasks expresse…
Chat-of-Thought: Collaborative Multi-Agent System for Generating Domain Specific Information
Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou +1
This paper presents a novel multi-agent system called Chat-of-Thought, designed to facilitate the generation of Failure Modes and Effects Analysis (FMEA) documents for industrial a…
LLM Assisted Anomaly Detection Service for Site Reliability Engineers: Enhancing Cloud Infrastructure Resilience
Nimesh Jha, Shuxin Lin, Srideepika Jayaraman +3
This paper introduces a scalable Anomaly Detection Service with a generalizable API tailored for industrial time-series data, designed to assist Site Reliability Engineers (SREs) i…
Towards Automated Solution Recipe Generation for Industrial Asset Management with LLM
Nianjun Zhou, Dhaval Patel, Shuxin Lin +1
This study introduces a novel approach to Industrial Asset Management (IAM) by incorporating Conditional-Based Management (CBM) principles with the latest advancements in Large Lan…