activity
20232025
most citedEvaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions

13 citations · 26 across the 8 of their papers we have counts for

collaborators

8 papers

cs.HC2025

The Invisible Mentor: Inferring User Actions from Screen Recordings to Recommend Better Workflows

Litao Yan, Andrew Head, Ken Milne +4

Many users struggle to notice when a more efficient workflow exists in feature-rich tools like Excel. Existing AI assistants offer help only after users describe their goals or pro…

cs.SE2025

TableTalk: Scaffolding Spreadsheet Development with a Language Agent

Jenny T. Liang, Aayush Kumar, Yasharth Bajpai +7

Spreadsheet programming is challenging. Programmers use spreadsheet programming knowledge (e.g., formulas) and problem-solving skills to combine actions into complex tasks. Advance…

cs.SE2024★ 2 cited

Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products

Nadia Nahar, Christian Kästner, Jenna Butler +3

Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous chall…

cs.AI2024★ 13 cited

Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions

Bhuvanashree Murugadoss, Christian Poelitz, Ian Drosos +5

LLMs-as-a-judge is a recently popularized method which replaces human judgements in task evaluation (Zheng et al. 2024) with automatic evaluation using LLMs. Due to widespread use…

cs.HC2024★ 1 cited

Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants

Bhavya Chopra, Yasharth Bajpai, Param Biyani +4

The widespread availability of Large Language Models (LLMs) within Integrated Development Environments (IDEs) has led to their speedy adoption. Conversational interactions with LLM…

cs.SE2023★ 6 cited

Building Your Own Product Copilot: Challenges, Opportunities, and Needs

Chris Parnin, Gustavo Soares, Rahul Pandita +3

A race is underway to embed advanced AI capabilities into products. These product copilots enable users to ask questions in natural language and receive relevant responses that are…

cs.HC2023★ 2 cited

Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities

Bhavya Chopra, Ananya Singha, Anna Fariha +4

Large Language Models (LLMs) are being increasingly employed in data science for tasks like data preprocessing and analytics. However, data scientists encounter substantial obstacl…

cs.CL2023★ 2 cited

Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMs

Ananya Singha, José Cambronero, Sumit Gulwani +2

Large language models (LLMs) are increasingly applied for tabular tasks using in-context learning. The prompt representation for a table may play a role in the LLMs ability to proc…