6 papers
Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education
Adam Coscia, Sujata Duwal, Langdon Holmes +2
LLMs are reshaping educational technology, yet evaluating their responses for pedagogical alignment remains underexplored, relying heavily on the expertise of learning engineers bu…
Semantic Bundling: Interactive Node and Edge Bundling to Simplify Knowledge Graphs using Large Language Models
Adam Coscia, Zeyu Hua, Eric Krokos +2
We present Semantic Bundling, a visual analytics technique for making sense of text documents represented as knowledge graphs (KGs). Representing a document corpus as a KG makes re…
LLM Prompt Evaluation for Educational Applications
Langdon Holmes, Adam Coscia, Scott Crossley +2
As large language models (LLMs) become increasingly common in educational applications, there is a growing need for evidence-based methods to design and evaluate LLM prompts that p…
VisPile: A Visual Analytics System for Analyzing Multiple Text Documents With Large Language Models and Knowledge Graphs
Adam Coscia, Alex Endert
Intelligence analysts perform sensemaking over collections of documents using various visual and analytic techniques to gain insights from large amounts of text. As data scales gro…
DraftMarks: Enhancing Transparency in Human-AI Co-Writing Through Interactive Skeuomorphic Process Traces
Momin N. Siddiqui, Nikki Nasseri, Adam Coscia +2
As generative AI becomes part of everyday writing, questions of transparency and productive human effort are increasingly important. Educators, reviewers, and readers want to under…
OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models
Adam Coscia, Shunan Guo, Eunyee Koh +1
As multi-turn dialogues with large language models (LLMs) grow longer and more complex, how can users better evaluate and review progress on their conversational goals? We present…