37 citations · 113 across the 14 of their papers we have counts for
14 papers
How Notations Evolve: A Historical Analysis with Implications for Supporting User-Defined Abstractions
Jingyue Zhang, J. D. Zamfirescu-Pereira, Elena L. Glassman +2
Traditional human-computer interaction takes place through formally-specified systems like structured UIs and programming languages. Recent AI systems promise a new set of informal…
Beyond Code Generation: LLM-supported Exploration of the Program Design Space
J. D. Zamfirescu-Pereira, Eunice Jun, Michael Terry +2
In this work, we explore explicit Large Language Model (LLM)-powered support for the iterative design of computer programs. Program design, like other design activity, is character…
Rambler in the Wild: A Diary Study of LLM-Assisted Writing With Speech
Xuyu Yang, Wengxi Li, Matthew G. Lee +3
Speech-to-text technologies have been shown to improve text input efficiency and potentially lower the barriers to writing. Recent LLM-assisted dictation tools aim to support writi…
Pensieve Discuss: Scalable Small-Group CS Tutoring System with AI
Yoonseok Yang, Jack Liu, J. D. Zamfirescu-Pereira +1
Small-group tutoring in Computer Science (CS) is effective, but presents the challenge of providing a dedicated tutor for each group and encouraging collaboration among group membe…
A Knowledge-Component-Based Methodology for Evaluating AI Assistants
Laryn Qi, J. D. Zamfirescu-Pereira, Taehan Kim +3
We evaluate an automatic hint generator for CS1 programming assignments powered by GPT-4, a large language model. This system provides natural language guidance about how students…
61A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?)
J. D. Zamfirescu-Pereira, Laryn Qi, Björn Hartmann +2
LLM-based chatbots enable students to get immediate, interactive help on homework assignments, but even a thoughtfully-designed bot may not serve all pedagogical goals. We report h…
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shreya Shankar, J. D. Zamfirescu-Pereira, Björn Hartmann +2
Due to the cumbersome nature of human evaluation and limitations of code-based evaluation, Large Language Models (LLMs) are increasingly being used to assist humans in evaluating L…
Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation
Susan Lin, Jeremy Warner, J. D. Zamfirescu-Pereira +8
Dictation enables efficient text input on mobile devices. However, writing with speech can produce disfluent, wordy, and incoherent text and thus requires heavy post-processing. Th…
SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines
Shreya Shankar, Haotian Li, Parth Asawa +7
Large language models (LLMs) are being increasingly deployed as part of pipelines that repeatedly process or generate data of some sort. However, a common barrier to deployment are…
Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets
Shm Garanganao Almeda, J. D. Zamfirescu-Pereira, Kyu Won Kim +2
Design space exploration (DSE) for Text-to-Image (TTI) models entails navigating a vast, opaque space of possible image outputs, through a commensurately vast input space of hyperp…
Detecting disparities in police deployments using dashcam data
Matt Franchi, J. D. Zamfirescu-Pereira, Wendy Ju +1
Large-scale policing data is vital for detecting inequity in police behavior and policing algorithms. However, one important type of policing data remains largely unavailable withi…
Conversation Regression Testing: A Design Technique for Prototyping Generalizable Prompt Strategies for Pre-trained Language Models
J. D. Zamfirescu-Pereira, Bjoern Hartmann, Qian Yang
Pre-trained language models (LLMs) such as GPT-3 can carry fluent, multi-turn conversations out-of-the-box, making them attractive materials for chatbot design. Further, designers…
Trucks Don't Mean Trump: Diagnosing Human Error in Image Analysis
J. D. Zamfirescu-Pereira, Jerry Chen, Emily Wen +3
Algorithms provide powerful tools for detecting and dissecting human bias and error. Here, we develop machine learning methods to to analyze how humans err in a particular high-sta…
Crowdsourcing County-Level Data on Early COVID-19 Policy Interventions in the United States: Technical Report
Jacob Ritchie, Mark Whiting, Sorathan Chaturapruek +5
Beginning in April 2020, we gathered partial county-level data on non-pharmaceutical interventions (NPIs) implemented in response to the COVID-19 pandemic in the United States, usi…