6 papers
IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurations
David Kaleko, Sergey Ivanov, Md Mofijul Islam
We present IDP AutoOpt, an autonomous LLM agent that discovers high-performing configurations for intelligent document processing (IDP) pipelines. Tuning IDP prompts, models, OCR s…
Loc2Repair: A Framework for Evaluating the Impact of File-Level Issue Localization in Repo-Level LLM Repair
Mohammad Nour Al Awad, Sergey Ivanov
Repository-grounded automated repair is often reported as a single end-to-end capability, which hides distinct failure modes such as poor file targeting, incorrect patch synthesis,…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Pre-Filtering Code Suggestions using Developer Behavioral Telemetry to Optimize LLM-Assisted Programming
Mohammad Nour Al Awad, Sergey Ivanov, Olga Tikhonova
Large Language Models (LLMs) are increasingly integrated into code editors to provide AI-powered code suggestions. Yet many of these suggestions are ignored, resulting in wasted co…
Optimizing LLM Code Suggestions: Feedback-Driven Timing with Lightweight State Bounds
Mohammad Nour Al Awad, Sergey Ivanov, Olga Tikhonova
Large Language Models (LLMs) have transformed code auto-completion by generating context-aware suggestions. Yet, deciding when to present these suggestions remains underexplored, o…
A Multimodal Conversational Agent for Tabular Data Analysis
Mohammad Nour Al Awad, Sergey Ivanov, Olga Tikhonova +1
Large language models (LLMs) can reshape information processing by handling data analysis, visualization, and interpretation in an interactive, context-aware dialogue with users, i…