1 citations · 2 across the 3 of their papers we have counts for
3 papers
ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks
Saurabh Jha, Rohan Arora, Yuji Watanabe +40
Realizing the vision of using AI agents to automate critical IT tasks depends on the ability to measure and understand effectiveness of proposed solutions. We introduce ITBench, a…
ScriptSmith: A Unified LLM Framework for Enhancing IT Operations via Automated Bash Script Generation, Assessment, and Refinement
Oishik Chatterjee, Pooja Aggarwal, Suranjana Samanta +8
In the rapidly evolving landscape of site reliability engineering (SRE), the demand for efficient and effective solutions to manage and resolve issues in site and cloud application…
CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation
Pooja Aggarwal, Oishik Chatterjee, Ting Dai +4
The advent of large language models (LLMs) has greatly facilitated code generation, but ensuring the functional correctness of generated code remains a challenge. Traditional valid…