8 papers
ASTRA - Agentic System for Ticket Resolution and Analysis
Shashidhar Reddy Javaji, Mohamed Trabelsi, Jin Cao +1
Technical operations teams resolve large volumes of incidents by synthesizing fragmented evidence from ticket text, historical cases, system logs, and technical documentation. Exis…
Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Prompting
Shashidhar Reddy Javaji, Bhavul Gauri, Zining Zhu
Large language models (LLMs) are now used in multi-turn workflows, but we still lack a clear way to measure when iteration helps and when it hurts. We present an evaluation framewo…
VERBA: Verbalizing Model Differences Using Large Language Models
Shravan Doda, Shashidhar Reddy Javaji, Zining Zhu
In the current machine learning landscape, we face a "model lake" phenomenon: Given a task, there is a proliferation of trained models with similar performances despite different b…
Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim Evidence Reasoning
Shashidhar Reddy Javaji, Yupeng Cao, Haohang Li +3
Large language models (LLMs) are increasingly being used for complex research tasks such as literature review, idea generation, and scientific paper analysis, yet their ability to…
FinAudio: A Benchmark for Audio Large Language Models in Financial Applications
Yupeng Cao, Haohang Li, Yangyang Yu +10
Audio Large Language Models (AudioLLMs) have received widespread attention and have significantly improved performance on audio tasks such as conversation, audio understanding, and…
INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent
Haohang Li, Yupeng Cao, Yangyang Yu +12
Recent advancements have underscored the potential of large language model (LLM)-based agents in financial decision-making. Despite this progress, the field currently encounters tw…