22 citations · 31 across the 10 of their papers we have counts for
12 papers
RubricCraft: Where Solutions Diverge, Criteria Emerge
Hosein Hasanbeig, Priyanshu Gupta, Ashish Tiwari +2
Evaluating language agents on open-ended, artifact-producing tasks is difficult: model-based graders can be inconsistent, while exhaustive programmatic checks are often impractical…
Whose turn is it? Context-guided model routing for stateful agents
Anikesh Parashar, Avik Dutta, Vashu Chauhan +6
Model routing can reduce the cost of LLM applications by assigning simpler requests to cheaper models without sacrificing the quality of the response. However, most routers treat r…
An Empirical Investigation of Robustness in Large Language Models under Tabular Distortions
Avik Dutta, Harshit Nigam, Hosein Hasanbeig +2
We investigate how large language models (LLMs) fail when tabular data in an otherwise canonical representation is subjected to semantic and structural distortions. Our findings re…
Distributions In, Distributions Out: The Case for Soft-Label Training
Agamdeep Singh, Ashish Tiwari, Hosein Hasanbeig +1
Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote. On tasks…
ConDABench: Interactive Evaluation of Language Models for Data Analysis
Avik Dutta, Priyanshu Gupta, Hosein Hasanbeig +6
Real-world data analysis tasks often come with under-specified goals and unclean data. User interaction is necessary to understand and disambiguate a user's intent, and hence, esse…
Progressive Safeguards for Safe and Model-Agnostic Reinforcement Learning
Nabil Omi, Hosein Hasanbeig, Hiteshi Sharma +2
In this paper we propose a formal, model-agnostic meta-learning framework for safe reinforcement learning. Our framework is inspired by how parents safeguard their children across…
Are LLMs Good Cryptic Crossword Solvers?
Abdelrahman Sadallah, Daria Kotova, Ekaterina Kochmar
Cryptic crosswords are puzzles that rely not only on general knowledge but also on the solver's ability to manipulate language on different levels and deal with various types of wo…
Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis
Rohan Mitta, Hosein Hasanbeig, Jun Wang +3
This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL), such that the safety constraint violations are bounded at any point during le…
Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications
Jun Wang, Hosein Hasanbeig, Kaiyuan Tan +2
This paper addresses the problem of designing control policies for agents with unknown stochastic dynamics and control objectives specified using Linear Temporal Logic (LTL). Recen…
Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Models
Safoora Yousefi, Leo Betthauser, Hosein Hasanbeig +2
Large language models (LLMs) exhibit remarkable performance improvement through in-context learning (ICL) by leveraging task-specific examples in the input. However, the mechanisms…
ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning
Hosein Hasanbeig, Hiteshi Sharma, Leo Betthauser +2
From grading papers to summarizing medical documents, large language models (LLMs) are evermore used for evaluation of text generated by humans and AI alike. However, despite their…
Evaluating Cognitive Maps and Planning in Large Language Models with CogEval
Ida Momennejad, Hosein Hasanbeig, Felipe Vieira +5
Recently an influx of studies claim emergent cognitive abilities in large language models (LLMs). Yet, most rely on anecdotes, overlook contamination of training sets, or lack syst…